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Executive summary 



The Saint Paul Public Schools’ Project Early 
Kindergarten program aims to improve the 
school readiness of Saint Paul children. The 
program offers a rigorous academic approach 
and targets children who are English Language 
Learners, come from low-income families, or 
need Special Education services. Ultimately, 
the program intends to help close Saint Paul’s 
achievement gap. 

The program began in 1 0 Saint Paul schools in 
fall 2005, and expanded to community child care 
settings a year later. Project Early Kindergarten 
(PEK) has since become the model for pre- 
kindergarten programs district-wide and is now 
titled the Saint Paul Public Schools’ Pre- 
Kindergarten Program. As of fall 2009, 29 
district elementary schools, 10 child care centers, 
and 13 family child care homes offer pre- 
kindergarten programs following the PEK 
approach. School sites offer the program to 4- 
year-olds, and child care sites to 214- to 4-year- 
olds. 

PEK aligns pre-kindergarten education with 
the district’s K-12 curriculum model, the 
Project for Academic Excellence. The model 
emphasizes standards-based education and 
extensive professional development. With 
sensitivity to young children’s developmental 
needs, PEK extends this model to early 
education, bringing children’s preschool 
experience into alignment with the educational 
experience they will have in later years. 

PEK is funded primarily by Saint Paul Public 
Schools and The McKnight Foundation, which 
provided an initial three-year grant in 2004 and 
renewed funding in 2007. PEK extends the 
program to child care settings through a 
partnership with Resources for Child Caring. 
The Minnesota Early Learning Foundation 



also had contributed funds to the child care 
portion of the program. 

Rigorous evaluation 

PEK participates in a rigorous, independent 
evaluation conducted by Wilder Research. 
Children are tested over time and in 
developmentally appropriate ways. Evaluators 
compare children’s academic and social skills in 
kindergarten and early elementary years to those 
of peers who did not participate in PEK. 

School results 

As of fall 2009, data are available for three 
cohorts of PEK school children. On average, these 
students experienced the following initial 
changes: 

■ In the year before kindergarten, all three 
groups of children who completed PEK in 
its first three years of operation made faster 
progress than their peers nationally in 
vocabulary and early reading and writing 
skills. The second group of children to 
complete PEK (Cohort 2) also made 
accelerated progress in early math, while the 
first and third groups to complete PEK 
(Cohorts 1 and 3) made expected progress. 

■ When they reached kindergarten, PEK 
children had academic skills that were 
substantially more advanced than those of 
similar, same-age children in a comparison 
group who had chosen but not yet received 
PEK. 

■ All three cohorts of PEK children also 
showed advantages compared to their 
kindergarten classmates, with differences 
tending to be stronger with each successive 
cohort. In all four academic areas assessed 
(vocabulary and early reading, writing, and 
math), Cohorts 2 and 3 scored significantly 
higher on average than both classmates with 
and classmates without prior preschool or 
child care center experience. 
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■ Teachers’ ratings of children in kindergarten 
also suggested that overall, PEK tended to 
enhance social skills, lessen problem 
behaviors, and improve academic 
competence more than other experiences 
that classmates had prior to kindergarten. 

■ Between fall of kindergarten and fall of first 
grade, the academic and social advantages 
that Cohort 1 and 2 children seemed to gain 
from PEK appeared to lessen somewhat on 
average, although PEK students continued 
to show academic advantages over 
classmates without preschool or child care 
center experience. In addition, children in 
Cohort 2 maintained advantages in early 
reading and writing skills over their 
classmates with preschool experiences. 

■ Principals, teachers, and parents provided 
very positive feedback about PEK. 

Child care results 

Having started a year later, PEK’s child care 
component is at an earlier stage. As of fall 
2009, data are available for 4-year-olds who 
participated in the child care component’s first 
and second cohorts. At this point, results are 
more suggestive than conclusive. On average, 
4-year-olds in child care Cohorts 1 and 2 
experienced the following changes: 

■ Upon kindergarten entry, PEK child care 
Cohort 1 and 2 children appeared to have 
an advantage over classmates who did not 
participate in PEK on some academic 
measures, especially vocabulary. 

■ However, PEK school-based children 
appeared to have a slight advantage over 
PEK child care children on reading and 
math in kindergarten. 

■ In the areas of social skills and problem 
behaviors, child care Cohort 1 and 2 
children did not appear to have any 
advantages compared to kindergarten 
classmates. Again, results tended to be 
more positive for PEK school children. 

■ Overall, child care center directors, center 
teachers, and family child care home 
providers gave positive feedback about 



their experiences with PEK and also 
offered some suggestions for further 
program development. 

Issues to consider 

A core component of PEK is the inclusion of 
an ongoing evaluation that can be used to inform 
programming. Based on results available to 
date, following are several issues that can be 
taken into consideration in future planning for 
PEK school and child care sites. 

A complete list of issues for consideration and 
“lessons learned” to date from the evaluation 
are provided in Wilder Research’s full report. 

■ The success of PEK in increasing the skills of 
participants results in skill differences between 
them and their classmates when they reach 
kindergarten. To ensure that all children are 
able to achieve substantial advances in 
kindergarten, it seems important that 
kindergarten instruction be differentiated to 
varying skill levels. Toward this end, PEK 
leaders began working intensively with four 
pilot schools to equip kindergarten teachers to 
differentiate their instruction based on 
children’s incoming skill levels. As the study 
continues, we will assess whether this effort 
impacts children’s academic skills. 

■ Particular attention may need to be paid to the 
socials skills and problem behaviors of children 
at child care sites. Teachers received training 
on Positive Behavior Support at the beginning 
of the 2008-09 school year. PEK staff can 
consider whether child care teachers could 
benefit from more training in this area. 

■ PEK child care results are limited at this point, 
suggesting some program impact on children 
but also suggesting room for improvement. We 
anticipate more reliable results in 2009-10 
when the larger third cohort of child care 
children begin kindergarten. Cohort 3 children 
are assessed both in the fall of their pre- 
kindergarten year and fall of their kindergarten 
year, permitting better analysis of program 
impact than in the first two cohorts. 
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Introduction 



Program background 



Overview 

Project Early Kindergarten (PEK) aims to improve the school-readiness of Saint Paul 
children and help close the achievement gap through offering high-quality educational 
experiences for preschool children. The program aligns Saint Paul’s pre-kindergarten 
education with the district’s K-12 curriculum model, the Project for Academic Excellence. 
In this way, the program brings children’s preschool experience close to the educational 
experience they will have in kindergarten and beyond. The program emphasizes standards- 
based learning, extensive professional development, and parent education and support. 
Because parents use a variety of care arrangements for their pre-kindergarten children, PEK 
promotes a community-wide approach involving both schools and child care programs. 

The program targets services to English Language Learners, low-income children, and 
children needing Special Education services. In practice, most participants also represent 
racial or ethnic minorities. Participating children either attend a half-day, five-day-a- 
week school year program at one of the participating Saint Paul schools, or receive 
similar curricular support at their child care center or family child care home. PEK 
schools began serving 4-year-olds in fall 2005, and child care programs extended the 
program to 214- through 4-year-olds in fall 2006. 

PEK sites 

Ten Saint Paul schools began offering PEK in fall 2005. These schools include Ames, 
Como Park, Dayton’s Bluff, Four Seasons, Hayden Heights, Maxfield, Prosperity 
Heights, Wellstone, and World Cultures/ American Indian Magnet, two schools which 
share a building and classroom. Since that time, PEK has become the model for all 
4-year-old programs district-wide with the exception of Montessori programs. As of fall 
2009, a total of 29 district elementary schools implement the PEK framework. 

PEK extends the program to child care settings through a partnership with Resources for 
Child Caring, a community agency working to improve the quality of early childhood 
care and education (Resources for Child Caring, n.d.). This community component of 
PEK is considered a pilot. The first cohort of partnering child care programs was asked 
to participate in PEK for two years, spanning the 2006-07 and 2007-08 school years. Six 
centers and 15 homes were originally selected to participate in the program. A second 
cohort of providers began offering PEK in fall 2008. They include 7 child care centers 
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that are new to PEK at that time, 1 continuing center, and 13 new family child care 
homes. As of fall 2009, all 8 child care centers and 10 of the 13 family child care homes 
are still offering PEK. In addition, three new family child care homes will join the 
second cohort of providers in 2009-10. 

Evaluation 

Wilder Research serves as the independent evaluator of PEK. The evaluation assesses the 
program at the 10 original school sites and at participating child care centers and family child 
care homes. For children attending at school sites, researchers use a quasi-experimental 
research design to assess impacts on children’s academic success. The study also follows 
school-based children into their early elementary years to see if program effects are 
sustained through early elementary school. Children attending at child care sites are 
assessed in kindergarten to allow for comparisons at that time to children who attended 
PEK school sites and children who did not attend PEK. Beginning in 2008, assessments 
are also conducted at child care sites with 4-year-old children. As with school cohorts, the 
third cohort of child care participants are assessed in the fall of their PEK year to facilitate 
measures of change between fall of PEK and fall of kindergarten. A complete description 
of research methods is provided in the Evaluation section of the report. 

Funding 

The program operates primarily through funding from Saint Paul Public Schools and The 
McKnight Foundation. In 2004 The McKnight Foundation provided a three-year, $2.8 
million grant for program development and implementation, and in 2007 McKnight 
contributed an additional $3 million for efforts through the 2009-10 school year. PEK 
extends the program to child care settings through a partnership with Resources for Child 
Caring. The Minnesota Early Learning Foundation also had contributed funds to the 
child care portion of the program in the 2007-08 and 2008-09 school years. 

In addition, PEK-Early Reading First, which operates under a federal grant, provides 
funds at two of the 28 PEK schools and two other child care centers. Wilder Research 
conducts a separate evaluation of PEK - Early Reading First program. Reports on the 
program’s first and second years are on Wilder Research’s website (see Mohr, Gozali- 
Lee, & Mueller, 2008a and Gozali-Lee, Broton, & Mueller, 2008). A report on the third 
year of PEK - Early Reading First will be prepared in fall 2009. 
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District pre-kindergarten consolidation 



In fall 2008, the Saint Paul Public Schools consolidated pre-kindergarten programs 
district-wide and detennined that all programs, except the Montessori programs, would 
use the PEK curricular approach. This consolidation unities five programs that 
previously operated separately. The consolidated program is titled the Saint Paul Public 
Schools’ Pre-Kindergarten Program. In this evaluation report, however, we still use the 
fonner program’s name, Project Early Kindergarten (PEK). In 2008-09, 28 district 
elementary schools offered pre-kindergarten classes. As of fall 2009, 29 elementary 
schools, with 35 classrooms and over 1,300 children participate in the PEK curricular 
framework. Following are the elements of consolidation adopted by the district: 

■ Classes meet five days a week for two and a half hours a day; 

■ Class times align with school start and end times to enable pre-kindergarten staff to 
participate in Professional Learning Communities and other school functions; 

■ Transportation is provided using the elementary school busing system (with separate 
busing provided for some Early Childhood Special Education children); 

■ Pre-kindergarten enrollment is processed by the district’s Student Placement Center; 

■ Class sizes are capped at 20 students; 

■ Classes are taught by a licensed teacher and an assistant teacher. Additional staff 
work in classrooms that include children with special education needs; 

■ Program management and staff supervision occur at the local school level under the 
direction of the principal, encouraging a team approach within the school; 

■ Early childhood professional development workshops and ongoing job-embedded 
coaching are standardized across programs; 

■ Using PEK’s Early Childhood Workshop framework, pre-kindergarten curriculum and 
instruction is aligned with the district’s Project for Academic Excellence elementary 
model, with a specific focus on aligmnent with kindergarten and first grade; 

■ Student, classroom, and program accountability measures are standardized; 

■ An Early Childhood Special Education (ECSE) inclusion model is maintained in 19 
of the 28 schools; 
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■ Parent education, family support, and student behavior support are provided district-wide; 

■ The Early Childhood Curriculum Resource center is made available district-wide; and 

■ Referendum funds are used to cover the cost of all pre-kindergarten general education 
teachers and assistants. The McKnight Foundation funds and School Readiness state 
aid are used for program support for all pre-kindergarten program schools. Special 
Education covers all ECSE teachers, assistants, therapists, and social workers. 

To ensure that gains made in its pre-kindergarten programs are sustained and built on in 
future years, the district is also working to connect pre-kindergarten with kindergarten 
teachers. Efforts are made to align programming during these early years and equip 
kindergarten teachers to differentiate instruction based on the varying needs of incoming 
students. For example, in 2008-09, PEK provided weekly coaching to kindergarten 
teachers in four schools (Dayton’s Bluff, Wellstone, American Indian, and World 
Cultures) to strengthen their capacity to differentiate instruction. To increase the 
connections with PEK teachers, the coach worked with both PEK and kindergarten 
teachers in Professional Learning Communities in these schools. 

Contents of the report 

This report comes at the conclusion of the fifth year of PEK. Following an initial 
planning year (2004-05), PEK has served children through the school component for four 
years (2005-06 to 2008-09) and through the community child care component for three 
years (2006-07 to 2008-09). This report summarizes the program’s implementation and 
outcomes results to date, through the 2008-09 school year. As shown in Figure 1, at this 
point Wilder Research outcomes data are available for children attending the first four 
years of PEK at school sites and the first three years of PEK at child care sites. This is an 
interim report, and future years’ data will be provided in subsequent reports. 
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1. Summary of outcomes data provided in this report 





Progress 
during PEK 


Fall of kindergarten 
results compared to 
peers 


Fall of 1 st grade 
results compared 
to peers 


School-based Cohort 1 

(PEK 2005-06) 


S 


V 


V 


School-based Cohort 2 

(PEK 2006-07) 




V 


V 


School-based Cohort 3 

(PEK 2007-08) 




V 


N/A° 


Community-based Cohort 1 

(PEK 2006-07) 




✓ b 


N/A d 


Community-based Cohort 2 

(PEK 2007-08) 




✓ b 


N/A d 


Community-based Cohort 3 

(PEK 2008-09) 


V* 


N/A c 


N/A d 



a Results of Individual Growth and Development Indicators administered to 4-year-olds by PEK staff are presented. For 

child care Cohort 3 only, the PPVT III and WJ III are also administered in fall of PEK (fall 2008) to children who will attend 
kindergarten the following fall. 



b Results reflect 4-year-olds who attended community-based PEK in 2006-07 ( Cohortl ) and in 2007-08 ( Cohort 2). 

c These data will be available following assessments conducted in fall 2009. 
d These data are not being collected. 

The report begins by describing PEK goals and components, followed by a section on 
evaluation methods. The main body of the report then summarizes evaluation results to 
date. Results are separated into two sections: one on the school component and one on 
the community child care component. Both sections summarize student outcomes as well 
as implementation results. The final section of the report explores the lessons learned 
thus far in the evaluation. These lessons will be modified and expanded as the evaluation 
continues, and are intended to provide information that may be instructive to the early 
childhood education community and policymakers. The report concludes with an 
Appendix of figures providing supplemental information. It should be noted that 
throughout this report, “teachers” is used to refer to school teachers, child care center 
teachers, and family child care home providers. 
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Program goals and components 



PEK’s goals include providing programming aligned with the district’s K-12 curriculum 
model and using a research-based approach to delivering services. Ultimately, the 
program intends to help close Saint Paul’s achievement gap. Key program components 
include aligmnent with the Project for Academic Excellence, involving extensive 
professional development; parent education and support; and participation in a rigorous 
evaluation. This section and the following section on evaluation describe these program 
goals and components as well as the program’s activities in these areas. 

Central goals 

PEK’s central goals, as stated by the program, follow: 

1. School-based: To develop optimal, developmental^ and academically focused early 
childhood programming aligned with the District’s K-12 standards-based comprehensive 
reform model, Saint Paul’s Project for Academic Excellence, for 4-year-old English 
Language Learner students, Special Education students, and students who qualify for 
free and/or reduced-price meals. 

2. Community-based: To use a research-based approach to deliver accurately targeted 
specialized services and support to early learners (primarily 3- and 4-year-old children), 
families, child care providers, and the greater local community that aligns with the 
district’s standards-based comprehensive refonn model and creates a smooth transition 
into kindergarten. 



Alignment with the Project for Academic Excellence 

With differences based on young children’s developmental needs, PEK brings children’s 
preschool experience into alignment with the educational experience they will have in 
kindergarten and beyond. This educational experience centers on the Project for Academic 
Excellence. The district introduced the Project for Academic Excellence in 2001 as a 
comprehensive academic reform model. Since that time, the Project for Academic 
Excellence has expanded from a pilot project in selected elementary schools to a district- 
wide approach implemented in every grade level. With the replication of PEK’s model 
across 4-year-old programs, instruction aligned with the Project for Academic Excellence 
now extends to early education district-wide as well. 
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The Project for Academic Excellence model emphasizes standards-based education and 
extensive professional development. It aligns the district’s curriculum model with state 
and national standards in reading, writing, math, and science. It also provides ongoing 
training for teachers and administrators based on national standards for effective training. 
Professional development includes best practices in standards-based instruction of core 
academic subjects. The model also emphasizes on-the-job coaching to help teachers develop 
lessons with clearly defined learning goals. Principals play an important role as instructional 
leaders who are involved in classrooms and oversee classrooms’ implementation of the 
model (Saint Paul Public Schools, 2005). 

In the district’s own language, following are the 10 core components of the Project for 
Academic Excellence (Saint Paul Public Schools, n.d.): 

1 . Standards-based curriculum and instruction as the foundation of reform; 

2. Extensive continuing professional development for teachers and administrators; 

3. Focus on a small number of core academic skills; 

4. Demonstration sites to promote replication; 

5. A shared sense of instructional leadership across the school and district; 

6. Content-based coaching of teachers, principals, and district leaders; 

7. Availability of essential materials for learning; 

8. Peer support for teachers; 

9. Standards-based assessment to monitor progress; and 

10. Increasing to scale across the district. 

Early Childhood Workshop 

Local and national experts in early childhood development and education developed a 
preschool curricular model for PEK aligned with the Project for Academic Excellence. 
This “Early Childhood Workshop” combines the Project for Academic Excellence’s 
Reader’s and Writer’s Workshops. Contributors included the district’s Reader’s and 
Writer’s Workshop professional development trainer and her consultant group, the 
California-based Foundation for Comprehensive Early Literacy Learning (CELL); the 
University of Minnesota’s Center for Early Education and Development; English Language 
Learner, School Readiness, and Special Education staff; and Project for Academic 
Excellence and PEK staff. 
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Materials are geared toward the developmental needs of young children and are based on 
best practices in early childhood education. They emphasize specific standards in personal 
and social development, language and literacy, mathematical thinking, and physical 
development and health. The Early Childhood Workshop model is presented in a 
comprehensive implementation manual for teachers. Manuals also provide infonnation 
on the Project for Academic Excellence and underlying Principles of Learning, PEK core 
content and early childhood standards, standards-based instruction, using standards-based 
assessment to monitor progress, and other topics relevant to program goals. Separate 
editions of the manual are provided to PEK school and child care teachers (Saint Paul 
Public Schools, 2007b). 

At school sites, licensed teachers use the implementation manual to develop lesson plans 
and integrate lesson themes throughout the classroom environment. Reflecting their 
unique needs and operations, child care centers use their manual in conjunction with 
Doors to Discovery, a complete literacy-focused curriculum. Family child care homes 
use their manual along with a theme-based curricular model developed specifically for 
them. Beginning in the 2007-08 school year (Cohort 3), school classrooms also 
implemented Everyday Mathematics, a curriculum used in district kindergarten through 
sixth-grade classes. 

Professional development 

Consistent with the Project for Academic Excellence, PEK emphasizes extensive ongoing 
professional development and on-the-job coaching for participating school and child care 
teachers. For school teachers, this training builds on the required educational credentials 
of teaching licenses and preschool certification. As an indication of the program’s 
investment in training, it supports three Resources for Child Caring coaches, five school 
coaches for pre-kindergarten and kindergarten classrooms, national literacy consultants, a 
“master coach” consultant, and a community and family specialist who promotes the 
program’s parent education efforts. At the beginning of the second grant period, the 
program also hired one additional part-time parent educator supported by the Minnesota 
Early Learning Foundation. The Program also supports a behavioral specialist and an 
assessment coach for the 2009-10 school year. 

PEK teachers attend an intensive training workshop at the beginning of the school year, 
spanning three days for school teachers and one or two days for child care teachers. During 
the year, school teachers meet in regular Professional Learning Communities and child care 
teachers attend monthly training meetings. Both school and child care teachers also 
participate in one-on-one weekly or biweekly coaching sessions. Program coaches, in turn, 
participate in master coaching sessions. School and child care teachers receive training on 
the following topics, for example: the role of rituals and routines; standards-based 
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instruction; progress monitoring to guide data-driven instruction; reading and writing 
strategies, including read alouds, shared reading, interactive writing, active learning, and 
guided oral reading; the Principles of Learning, which underlie the Project for Academic 
Excellence; Positive Behavior Support; differentiated instruction; components of Classroom 
Assessment Scoring System (CLASS) instrument; and parent education. PEK also arranges 
for school and child care teachers and school principals to visit other PEK sites. 

As part of the district’s efforts for furthering the connections between pre- kindergarten 
and kindergarten classrooms, in the 2008-09 school year kindergarten teachers received a 
two-day workshop in differentiated instruction and in many schools, pre-kindergarten and 
kindergarten teachers worked together in Professional Learning Communities. At PEK 
schools, more intensive training was also offered to kindergarten teachers at Dayton’s 
Bluff, Wellstone, American Indian, and World Cultures during the 2008-09 school year. 
Kindergarten teachers in these schools received one-on-one weekly coaching sessions. 

The coaches also worked with both kindergarten and PEK teachers in Professional 
Learning Communities. In 2009-10, coaching and training workshops will be offered to 
all kindergarten teachers on a voluntary basis. 

Professional development is also provided to school principals and child care center 
directors and assistant directors to equip them to assume the role of the instructional 
leader at their school or center. Child care center directors receive six months of monthly 
training before their teachers begin working with the program. 

Principals and center directors as instructional leaders 

A tenet of the Project for Academic Excellence is that principals assume the role of the 
instructional leader at their school. Likewise, principals at PEK schools and directors at 
participating child care centers assume the role of the instructional leader of PEK at their 
site. This role provides site-level accountability for fidelity with the program model. At 
schools, the role also facilitates PEK’s integration into the school as a whole. The 
program places a strong emphasis on developing linkages between PEK, kindergarten, 
and early elementary teachers as a way of ensuring smooth transitions for students and 
curricular alignment across grade levels. The role is new for child care centers as of fall 
2008, and is intended to equip center directors to provide initial training to new teachers 
who start after the intensive training workshop at the beginning of the year. 

School principals and center directors receive professional development to prepare them 
for assuming this role. Program coaches also provide them with memos to guide them in 
making classroom observations. These memos describe instructional best practices from 
the latest professional development teachers have received that should be evident in the 
classroom. Since fall 2007, program administrators, principals, and child care center 
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directors have also conducted “Progress Monitoring Walks” to check fidelity of program 
implementation. 

Progress monitoring 

The Project for Academic Excellence emphasizes ongoing progress-monitoring. PEK 
teachers use developmentally appropriate tools to monitor progress in children’s skills and 
their growth toward developmental milestones. Work Sampling System assessments and 
Individual Growth and Development Indicators (IGDIs) help teachers understand changes 
in individual children and alert them when a child may require more intensive 
interventions. Additionally, starting in fall 2008, the PEK teachers at school sites began 
using some subtests of the Phonological Awareness Literacy Screening (PALS), the 
instrument used in the district’s K-3. As with their K-12 counterparts, PEK teachers use 
information gathered through the ongoing assessments to inform their instruction. 

Parent education and support 

PEK also emphasizes parent involvement in their children’s learning as well as parent- 
school connections. PEK supports work to increase parents’ understanding of the skills 
children need for school, and parents’ engagement with their children in literacy activities 
at home. They also aim to help parents feel comfortable navigating the school system 
and participating in school activities. Parent-education efforts are coordinated by the 
program’s community and family specialist as well as a part-time parent educator who 
works to connect child care families with neighborhood schools. 

PEK developed extensive parent-education materials, titled “School and Home - Partners 
in Learning,” that were implemented in 2007-08. Materials include literacy activities that 
parents can do with children at home. Math activities were added in the 2008-09 school 
year. Every week, parents also receive take-home infonnation in different languages that 
reinforces skills being taught in PEK and explains how to use the literacy and math 
materials. Parents also receive information about community resources. To facilitate 
home learning over the summer, teachers also distributed summer writing kits to PEK 
school and child care children who were going on to kindergarten. 

In addition to developing parent-education materials, PEK offers parenting events and 
parent-education sessions at the schools, and brings school services to child care centers. 
For example, the program offers parent orientations at the schools and provides welcome 
packets with infonnation about transitioning to school. As another example, PEK provides 
“Understanding School Choice” sessions at participating child care centers during which 
district student placement staff answer parents’ questions and help parents register their 
children for kindergarten and Early Childhood Screening. 
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Evaluation 



PEK participates in a rigorous evaluation. The program views evaluation as an important 
sustainability strategy in that ultimately, the evaluation will provide evidence of whether 
the model warrants continuation and replication. The evaluation includes two components: 
an implementation evaluation and an outcomes evaluation. Wilder Research holds 
primary responsibility for the evaluation, with support and assistance from Saint Paul 
Public Schools’ Department of Research, Evaluation and Assessment. 



Program implementation 

The implementation evaluation addresses the overarching question, Does PEK provide a 
high-quality preschool program that is aligned with the Project for Academic Excellence 
and integrated into the school system? The implementation evaluation also assesses the 
degree to which PEK is serving the target population of high-need students, as well as 
parent involvement and school-family linkages. 

Researchers gather information on the children served and the extent to which schools 
and child care settings are implementing the program. Information is gathered from 
surveys and focus groups conducted by Wilder Research, records data provided by the 
district and PEK staff, and observations conducted and reports prepared by the program’s 
evaluator from Saint Paul Public Schools and staff of the University of Minnesota’s 
Center for Early Education and Development. Principal and PEK teacher surveys 
provide information on principals’ perceptions of PEK implementation and teachers’ 
interactions with parents. The kindergarten teacher survey gives information on their 
connections with PEK and its teachers. Parent surveys provide infonnation on their 
involvement in their children’s learning and school activities, their satisfaction with PEK, 
and children’s prior educational experiences and family background. Focus groups with 
child care teachers and directors provide feedback on their experiences with the program. 
To gather information about how the program is implemented in each setting, outside 
observers use structured questionnaires. Additionally, school and program records 
provide information about student enrollment, demographics, and attendance at PEK. 
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Program outcomes 



Wilder Research’s evaluation focuses on the program’s outcomes. It answers the key 
question, Does a high-quality preschool program aligned with the Project for Academic 
Excellence improve students’ educational outcomes? To answer this, evaluators need to 
know the following: 

■ Are children better prepared for kindergarten because they participated in PEK? 

■ Do they perform better in elementary school (kindergarten through third grade)? 

■ What are the benefits for children, families, and teachers of having pre-K programs 
integrated with schools? 

■ Is it cost-effective? 

Wilder Research addresses these questions through a quasi-experimental research design. 
Children are tested over time and in developmentally appropriate ways to see how they 
progress academically and socially, and whether program effects are sustained through 
early grade school. The study compares a treatment group of children who received PEK 
services with a comparison group who did not. Experimental research, involving random 
assignment to treatment and control groups, can be difficult to attain in education research. 
This quasi-experimental approach presents a rigorous alternative. While the study will not 
be able to prove absolutely that PEK causes specific outcomes, researchers will be able to 
draw reasonable inferences about the changes that can be attributed to the program. 

The study’s design and its use of nationally validated assessment instruments also allow 
researchers to compare PEK results with results from other public school-related preschool 
programs around the country. The Peabody Picture Vocabulary Test III (PPVT III) 
measures receptive vocabulary, and three subtests of the Woodcock-Johnson III Tests of 
Achievement (WJ III) measure early skills in reading, writing, and math. Wilder Research 
staff administer these tests one-on-one with children at the school sites each fall, and 
beginning in 2008-09, with children at child care sites. Teachers also complete 
assessments of individual students in the fall. They assess students’ social skills, problem 
behaviors, and academic competence on the Social Skills Rating System (SSRS). Figure 
2 provides the study’s assessment schedule over the five-year period from 2005-06 to 
2009-10. More detailed infonnation about the school and child care portions of the study 
are provided following the figure. 
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2. PEK assessment schedule, 2005-06 to 2009-10 



Fall 

Groups 2005 


Fall 

2006 


Fall 

2007 


Fall 

2008 


Fall 

2009 


Spring 

2010 


SCHOOL COMPONENT 
Cohort 1: 

PEK PEK 

students 


Kindergarten 


First grade 


None 0 


None 


Third 

grade d 


Classmates 3 None 


Kindergarten 


First grade 


None 0 


None 


Third 

grade d 


Cohort 2: 

PEK 

students 


PEK 


Kindergarten 


First grade 


Second 

grade 


None 


Classmates 3 


None 


Kindergarten 


First grade 


Second 

grade 


None 


Cohort 3: 

PEK 

students 




PEK 


Kindergarten 


First grade 


None 


Classmates 3 




None 


Kindergarten 


First grade 


None 


COMMUNITY (CHILD CARE) 
COMPONENT 

Cohort 1 


None b 


Kindergarten 


None 


None 


None 


Cohort 2 




None b 


Kindergarten 


None 


None 


Cohort 3 






PEK b 


Kindergarten 


None 


Cohort 4 








PEK b 


None 



a “Classmates” refers to the comparison group students who attended kindergarten at the 10 original PEK schools and who did not attend PEK at school 
or child care sites. 

b Individual Growth and Development Indicators (IGDIs) are used in PEK child care. For child care Cohorts 3 and 4 only, the PPVT III and WJ III are also 
administered in fall of PEK (fall 2008 and fall 2009) to children who will attend kindergarten the following fall. 

c Cohort 1 school students who participated during the program’s initial year of implementation are not assessed in second grade. 

d MCA-lls in reading and math plus SSRS. 

Note: Unless otherwise noted, this assessment schedule pertains to the WJ III, PPVT III, and SSRS. If funding permits, Cohorts 2 and 3 at PEK 

school sites will also be followed into third grade (2010-1 1 for Cohort 2 and 201 1-12 for Cohort 3). Cohort 4 at child care sites will be followed into 
kindergarten in fall 2010. 
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PEK school sites 

For children attending the 10 original PEK schools, the study assesses the following 
program outcomes: 1) the progress they make during PEK, and 2) the impact of PEK on 
their later academic performance. Progress during PEK is measured by comparing 
children’s baseline (fall of PEK) test scores with their scores one year later, in the fall of 
kindergarten. To measure PEK’s impact, the study compares PEK participants’ academic 
and social skills to those of their peers over time, as described below. 

Comparisons to peers 

Using the assessments mentioned earlier, children attending PEK schools are compared 
to two different groups of peers. First, they are compared to similar children who applied 
and were accepted for PEK, but who have not yet attended the program. In this analysis, 
children who just finished PEK constitute the “treatment” group, and children who are 
just beginning PEK constitute the “no-treatment” comparison group. Because children 
develop rapidly at this age, Wilder Research uses a statistical model that estimates the 
difference between the two groups right at the program’s September 1 birthday cutoff 
point. Near the cutoff point, children from both groups are essentially the same age but 
treatment-group children have completed the program and comparison-group children 
have not. This analysis provides a comparison of children with similar characteristics, 
and eliminates the selection bias that can occur if families who choose to enroll their 
children in the program differ in important ways from those who do not. This analysis is 
referred to as the “birthday cutoff’ method, illustrated in Figure Al. 

Second, once PEK children reach kindergarten, they are compared to their kindergarten 
classmates. These classmates may differ in some ways from PEK children. They have had a 
range of prior preschool and child care experiences, and some have had no fonnal preschool 
or child care experiences at all. This comparison reveals how developmental skills of PEK 
children compare to skills of kindergartners coming from a variety of backgrounds. 

Comparisons over time 

To see whether program effects last over time, PEK school children and their classmates are 
assessed in subsequent years as well. The study will continue to follow these two groups 
through third grade, as funding pennits. The same assessments of academic and behavioral 
progress described earlier will be used in these early primary grades, with the exception of 
third grade when the Minnesota Comprehensive Assessments-Series II will be used. 

It should be noted that the classmate comparison group is defined as children who: a) are 
kindergarten classmates of fonner PEK children, and b) attend kindergarten at one of the 



Project Early Kindergarten evaluation: 
Results through 2008-09 



16 



Wilder Research, September 2009 





10 original PEK school sites. PEK children are followed in kindergarten as long as they 
remain in any public (including charter) or private school in Saint Paul. After kindergarten, 
both the former PEK school students and the comparison group are followed as they move 
through the primary grades as long as they remain in schools in Saint Paul. 

PEK child care sites 

In the child care component, the evaluation of program outcomes is similar to but not as 
extensive as the evaluation of the school-based component. Wilder Research assesses 
academic progress during the PEK year for children in child care Cohorts 3 and 4, 
assessing them in both the fall of PEK and the fall of kindergarten. Children in child care 
Cohorts 1 and 2 were assessed in kindergarten only. For all four child care cohorts, 

PEK’s impact will be assessed in fall of kindergarten but not later years. In kindergarten, 
evaluators compare PEK child care participants’ academic and social skills to those of 
their kindergarten classmates and those of students who attended the PEK school 
component. These comparisons are based on the same assessments used in the school 
component (i.e., the PPVT III, WJ III, and SSRS). 

Other measures 

In addition to the child assessments conducted as part of the evaluation, teachers also use 
formal tools to monitor individual children’s progress over the course of the year. These 
tools include Work Sampling System assessments and Individual Growth and Development 
Indicators (IGDIs). Beginning in the 2008-09 school year, teachers also use some 
subtests of the Phonological Awareness Literacy Screening (PALS). Although not 
formally a part of the evaluation, the IGDI results are discussed briefly in the context of 
other student outcomes presented in this report. Finally, once sufficient data are 
available, Wilder Research’s economists plan to conduct a cost-effectiveness analysis of 
the program. The analysis will be based on placing PEK findings in the context of other 
studies following participants over longer periods of time. 

Statistical significance 

In some cases, this report refers to differences between groups that are “significant.” By 
significant, we mean that the difference is significant at the 0.05 level based on a statistical 
test. In other words, there is less than a 1 in 20 probability that the difference occurred by 
chance. 
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Progress summary: School-based PEK 



This section provides results available to date for the 10 original PEK schools. The 
section begins by profiling children who attended PEK schools during the program’s first 
three years, 2005-06 (Cohort 1), 2006-07 (Cohort 2), and 2007-08 (Cohort 3). Their 
progress during PEK is then discussed, based on Wilder Research’s assessments. 
Academic and social outcomes are then provided for the three cohorts based on Wilder 
Research’s assessments. After summarizing student outcomes, this section briefly 
describes program implementation through 2007-08. As evaluation follows former PEK 
children who are in kindergarten and first grade in 2008-09, evaluation in this area includes 
principals’ and kindergarten teachers’ perceptions on the impacts of PEK in their schools and 
on the connections between PEK and kindergarten classrooms. The section concludes with a 
list of issues for consideration that can be used to inform ongoing program planning efforts. 

More specifically, this section addresses the following topics for the school component: 

■ Overview of results 

■ Characteristics of children (Cohorts 1-3) 

■ Progress while in PEK (Cohorts 1-3) 

■ Kindergarten readiness compared to similar children (Cohorts 1-3) 

■ Kindergarten readiness compared to classmates (Cohorts 1-3) 

■ Differences in first grade compared to classmates (Cohorts 1 and 2) 

■ Implementation efforts (Cohorts 1-3) 

■ Issues for consideration 

Overview 

Results show promising progress for children attending PEK schools in 2005-06 (Cohort 1), 
2006-07 (Cohort 2), and 2007-08 (Cohort 3). All cohort children showed academic and 
social advantages over peers when they reached kindergarten. Children’s academic gains 
made during the pre-kindergarten year have also increased with each successive cohort. 
This trend may be associated with the development of PEK. That is, as PEK has become 
more fully implemented and mature as a program, its impact may have increased 
correspondingly. By first grade, differences between PEK students and their kindergarten 
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classmates had narrowed for the first two groups of PEK students to reach first grade. Data 

gathered over the next few years will help researchers make stronger claims about the 

program’s initial and longer-tenn impacts. 

On average, children in the initial school cohorts experienced the following changes: 

■ In the year before kindergarten, all three PEK cohorts made faster progress than their 
peers nationally in vocabulary and early reading and writing skills. Cohort 2 also 
made accelerated progress in early math skills, while Cohorts 1 and 3 made expected 
progress. 

■ When they reached kindergarten, PEK children had academic skills that were 
substantially more advanced than those of similar, same-age children in a comparison 
group who had chosen but not yet received PEK. 

■ All three cohorts showed advantages compared to their kindergarten classmates, and 
the differences tended to be stronger with each successive cohort. In all four academic 
areas assessed (vocabulary and early reading, writing, and math skills), Cohorts 2 and 
3 scored significantly higher on average than both classmates with and classmates 
without prior preschool or child care center experience. 

■ Teachers’ ratings of children in kindergarten also suggested that, overall, PEK tended 
to enhance social skills, lessen problem behaviors, and improve academic competence 
more than other experiences that classmates had prior to kindergarten. 

■ Between fall of kindergarten and fall of first grade, the academic and social advantages 
that children in Cohorts 1 and 2 seemed to gain from PEK appeared to lessen somewhat 
on average. PEK students made less progress than their classmates did on average 
between kindergarten and first grade, narrowing the gap between the groups. Nevertheless, 
PEK students continued to show academic advantages over classmates without preschool 
or child care center experience. In addition, children in Cohort 2 maintained advantages in 
early reading and writing skills over their classmates with preschool experience. 

Key evaluation findings to date also include the following: 

■ Compared to publicly funded pre-kindergarten programs in several other states, the 
estimated effect of PEK tended to be larger based on results of the three cohorts in 
vocabulary and early writing skills, and the first two cohorts in vocabulary and early 
writing and math skills. Early reading skill results are comparable to the other 
studies. 
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■ PEK school principals, teachers, and parents provided very favorable feedback about 
the program. 

■ Overall, structured classroom observations found that PEK classrooms have achieved 
a high level of alignment with the Project for Academic Excellence and are strong in 
their intentional supports for language and literacy. 



Characteristics of children 

Ten Saint Paul elementary schools began offering PEK to 4-year-olds in fall 2005. 
Between morning and afternoon sessions, these schools have the capacity to serve a total 
of 360 PEK children. Figure 3 shows the number of children in the three cohorts at PEK 
school sites. It is important to note that these numbers reflect most but not all children 
who have participated in the program. Wilder Research defines each cohort as those who 
are assessed in fall of their PEK year, and there have been some participants who were 
not assessed as part of the study. Some children were not assessed because they started 
the program later in the year, left the program in the fall, transferred schools, were absent, 
or did not have parental permission to participate in the assessments. 



3. Children attending PEK school sites, 2005-06 to 2007-08 


Cohort 


Number of 
children 


Cohort 1 (PEK 2005-06) 


326 


Cohort 2 (PEK 2006-07) 


329 


Cohort 3 (PEK 2007-08) 


312 


Total 


967 



Note: A total of 360 children can be served by the 10 PEK schools. Wilder Research defines each cohort as children 

who were assessed as part of the study in fall of their PEK year. As explained in the text , this definition includes most but not 
all children who have participated in the program. Numbers in this figure may also differ slightly from those in other figures in 
this and other PEK reports depending on the inclusion or exclusion of children tested in Spanish , children whose birth date 
was outside the range for their cohort based on the program ’s birthday cutoff date , and children completing only the Peabody 
or the Woodcock-Johnson but not both. There may also be variations based on missing data for some variables. 



Demographics 

Figure A2 in the Appendix provides demographic profiles of students in school-based 
Cohorts 1, 2, and 3. Some demographic characteristics can change over time, and these 
profiles reflect demographic data from fall of the PEK year. In each year, a majority of 
PEK students were low-income (61-74%), defined here as eligible for free- or reduced- 
price lunch. Just under half were English Language Learners (45-49%). Among those 



Project Early Kindergarten evaluation: 
Results through 2008-09 



20 



Wilder Research. September 2009 





with a primary home language other than English, Hmong was the most common home 
language followed by Spanish. More than 1 in 10 children in each cohort needed Special 
Education services (11-12%). Looking at these three categories together, 79-88 percent 
were in the program’s target population across the three years, meaning they were either 
low-income, English Language Learners (ELL), or needed Special Education services. 
Additionally, most students were from racial or ethnic minorities (81-85%). Eigure 4 
depicts the representation of PEK’s target populations in the first three cohorts. 



4. PEK school component. Representation of PEK target populations, 2005- 
06 to 2007-08 



88% 87% 




□ ELL ^Low-income "Special Education □ Target population 



Note: PEK targets children who are English Language Learners (ELL), from low-income families, or need Special 

Education services. “Target population” reflects the percentage of children who are in any of these three groups. 



Comparison group demographics 

Demographic characteristics of the classmate comparison groups are presented in the 
Appendix. The demographic infonnation reported is based on the infonnation provided 
by the district in kindergarten. As noted in Figures A3-A5, there were some differences 
between cohort children and their classmates. For example, there were more children in 
comparison groups than in Cohort 1 and 3 who were eligible for free or reduced-price 
lunch. In cases where former PEK students differed in meaningful ways from the 
comparison groups, we statistically adjusted for those demographic differences in our 
analysis. We also adjusted for any differences among the groups based on when in the 
fall they were tested. 
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Changes over time 

It is important to note that in some cases, children’s demographic characteristics can 
change over time. For example, it may not be known that a child needs Special Education 
services until after that child has been in the school system. As another example, a child 
may be eligible for free or reduced-price lunch one year and ineligible another year. 
Additionally, methods for obtaining PEK children’s demographic characteristics changed 
in 2006 after the district introduced a new application process for 4-year-old programs 
that collects applicants’ demographic infonnation. 

Changes due to attrition 

Demographics presented here reflect all students in the original PEK cohorts. However, 
attrition occurs over time in the study. Subsequent years’ analyses reflect only those 
students who were tested in a given year. Children attending PEK at school sites are 
followed after their PEK year as long as they remain in schools in Saint Paul. Children 
attending kindergarten or first grade outside of Saint Paul are not reflected in analyses 
presented in this report for fall of those years. Attrition also occurs in the comparison 
groups. Comparison groups are defined as kindergarten classmates of PEK children at 
the 10 original PEK schools. After kindergarten, comparison group students are followed 
as long as they remain in schools in Saint Paul. 

Figure 5 shows the number of PEK and comparison group children who were assessed in 
fall 2008. At that time, Wilder Research conducted assessments with 878 children who 
attended PEK school sites in Cohorts 2 and 3, and their comparison groups. Based on the 
numbers in Figure 3, we were able to assess 71 percent of the original Cohort 2 children 
when they were in first grade, and 75 percent of the original Cohort 3 children when they 
were in kindergarten. Cohort 1 children were not assessed in the fall 2008. 
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5. PEK school component. Fall 2008 study groups 



Study groups 


Number 

assessed 


Cohort 1 (PEK 2005-06) 


None® 


Cohort 1 comparison 3 


None® 


Cohort 2 (PEK 2006-07) d 


232 


Cohort 2 comparison 13 


181 


Cohort 3 (PEK 2007-08) 


235 


Cohort 3 comparison 0 


230 


Total 


878 



a Kindergarten classmates of PEK school -based Cohort 1 children in 2006-07 at the 10 PEK schools. 

b Kindergarten classmates of PEK school -based Cohort 2 children in 2007-08 at the 10 PEK schools. 

c Kindergarten classmates of PEK school -based Cohort 3 children in 2008-09 at the 10 PEK schools. 

d Children who attended both PEK at school sites and at child care sites are included in the school-based component 

numbers (six children in Cohort 2). 

e Cohort 1 and comparison classmates were not assessed in fall 2008. Their progress will be reported again following 
spring 2010 assessments. 

We compared the fall of PEK (baseline) demographics of these children to those of 
children who remained in the study to see if they differed in important ways. For the 
second and third cohorts, those assessed in fall 2008 appeared to resemble the original 
cohort. 

Home life 

Most PEK school children participating the first three years lived with both parents 
(70-73% in Cohorts 1, 2, and 3), and more than 1 in 10 lived with their mother only 
(15-17%). Quite frequently other adult relatives also lived in the household. A majority 
of children’s parents graduated from high school or attended some college but did not 
receive a four-year degree (67-69% of mothers and female caretakers, and 63-68% of 
fathers and male caretakers) in Cohorts 1, 2, and 3. 

School experience 

Children often enrolled in PEK without any prior preschool or child care experience. 
About 4 in 10 attended preschool, Head Start, or a child care center before they started 
PEK (36-40% in Cohorts 1, 2, and 3). Children also were typically not in another 
preschool or child care program while they attended PEK. When not in their PEK class, 
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children were most commonly cared for by parents (45-47% in Cohorts 1, 2, and 3). 
Other common arrangements involved - sometimes in combination with parental care - 
care from relatives, neighbors, or friends. 



Progress while in PEK 

For each cohort, progress during their PEK year is measured by comparing their fall of 
PEK (baseline) test scores with their fall of kindergarten test scores. Comparisons are 
made based on the Peabody and Woodcock-Johnson academic assessments conducted by 
Wilder Research. Because children develop rapidly at this age, we looked at how their 
progress compared to how much children of this age would be expected to progress based 
on national norms. 

Academic progress compared to national peers 

Figure 6 depicts PEK students’ progress during the pre-kindergarten year, shown 
separately for each of the three cohorts. The analysis is based on test scores that are age- 
standardized. This means that no change in scores from one year to the next indicates 
normative progress, positive change indicates accelerated progress, and negative change 
indicates slower progress in comparison to one’s peers nationally. PEK school-based 
students made substantial gains in academic skills during their PEK year. Compared to 
their peers nationally, students in all three cohorts made accelerated progress in 
vocabulary, early reading, and early writing. In other words, on average they made faster 
progress over the course of the year in these areas than did their peers nationally. 

Progress in early math skills differed by cohort. Whereas students in Cohorts 1 and 3 
made nonnative progress in math during their PEK year, Cohort 2 students made 
accelerated progress in this area compared to their peers nationally. Still, progress in 
math lagged behind progress seen in the other three academic areas (Figure A6; Mueller, 
2008). It should be noted that math was not a focus during the program’s first two years 
of implementation. The program implemented the Everyday Mathematics curriculum in 
the fall of Cohort 3’s PEK year (2007). Nevertheless, on average Cohort 3 students did 
not make accelerated progress in math during their PEK year (average score increased 
somewhat, but the difference was not statistically significant). 

Despite their substantial gains in academic skills, on average fonner PEK students were 
somewhat below national norms in vocabulary and early math skills in fall of their 
kindergarten year. This does not seem surprising given the program’s large ELL 
population and that math was not a focus during the program’s first two years. On the 
other hand, PEK students were slightly above national norms in early reading and writing 
skills in the fall of their kindergarten year, on average. 
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It is also worth noting that the overall average gains made during the PEK year have 
increased with each successive cohort. In other words, the second cohort of PEK children 
made stronger gains than the first cohort, and the third cohort of PEK children made 
stronger gains than both of the previous cohorts. In addition, students’ scores in fall of 
kindergarten have been slightly higher each year, indicating a higher level of kindergarten 
readiness for each successive cohort. At the same time, it should be noted that the average 
number of days between the fall of kindergarten and fall of preschool testing periods has 
varied somewhat by cohort, ranging from 375 days for Cohort 2 to 390 days for Cohort 1 
and 435 days for Cohort 3. This should be taken into account when comparing results 
across the cohorts because children who had a longer gap between testing may have 
progressed more in part because they had more time to develop. 

Academic progress in age-equivalency terms 

Translating results into age-equivalency scores provides another meaningful way of 
looking at these changes. In vocabulary, Cohort 3 children were estimated to be at 3 
years 6 months in the fall of PEK on average, and at 5 years 6 months in the fall of 
kindergarten on average, for a 24-month gain. This compares to an average vocabulary 
gain of 18 months for Cohort 2 children and 15 months for Cohort 1 children. In early 
reading skills, Cohort 3 children were estimated to have experienced a 19-month gain 
during their pre-kindergarten year on average, compared to a 16-month gain for Cohort 2 
and a 14-month gain for Cohort 1. Similarly, children in Cohort 3 made larger average 
gains in early writing (22 months) compared to children in Cohorts 1 and 2(17 months). 
It should be noted, though, that age-equivalency scores are a less exact measure than 
standard scores, which are used in other analyses presented here. For this reason, in age- 
equivalency tenns it appears that Cohort 2 children made the same size gains as Cohort 1 
children in early writing (17 months) and math (12 months), even though Cohort 2 
children had larger gains in these areas using standard scores. In addition, it appears that 
children in Cohort 3 made larger gains in math compared to children in Cohort 2 (16 vs. 
12 months), when the standard score results show the opposite (Figure A7; Mueller, 
2008). Again, it should also be noted that the average number of days between the 
testing periods has varied somewhat across the cohorts. 

Variations in academic progress among demographic groups 

PEK students’ progress during the pre-kindergarten year was examined within the 
specific demographic groups targeted by the program (Mueller & Gozali-Lee, 2007; 
Mueller, 2008). The results show that, on average, ELL students made significantly 
larger gains than non-ELL students in some areas, including vocabulary for Cohorts 1 
and 3, math for Cohorts 1 and 2, and reading for Cohort 1 only. Students in Special 
Education made significantly less progress than other students in reading for Cohorts 1 
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and 3 and in writing for Cohort 1 only. In contrast, students in Special Education made 
significantly larger gains than other students in math for Cohort 2. Results for Cohort 1 
indicate that students who were eligible for free or reduced-price lunch made 
significantly larger gains than students who were ineligible in the areas of vocabulary and 
reading. However, results for Cohorts 2 and 3 showed no significant differences on this 
income measure in any of the four areas. Lastly, the results suggest some significant 
differences based on race/ethnicity, although the findings are not very consistent across 
the cohorts and measures. The most consistent finding is that Asian students made 
significantly larger gains than some other racial/ethnic groups, most frequently Caucasian 
students. This result was found for Cohorts 1 and 3 in the areas of vocabulary and math, 
and for Cohort 1 only in the area of reading (Figures A8-A1 1 for Cohort 3 and Mueller, 
2008 for Cohorts 1 and 2). 



Project Early Kindergarten evaluation: 
Results through 2008-09 



26 



Wilder Research, September 2009 






Project Early Kindergarten evaluation: 
Results through 2008-09 



27 



Wilder Research, September 2009 





Kindergarten readiness compared to similar children 

Kindergarten readiness is assessed in two ways: 1) by comparing PEK children to similar 
children who applied and were selected for PEK but who have not yet participated, and 
2) by comparing PEK children to their kindergarten classmates. This section discusses 
kindergarten readiness compared to similar children selected for PEK. The comparison is 
based on the Peabody and Woodcock- Johnson academic assessments conducted by 
Wilder Research. 

Using the “birthday cutoff’ method (see Figure Al), children who just finished PEK are 
compared to children who are just beginning the program. An advantage of this analysis is 
that it minimizes the selection bias that could occur if there were differences between 
families who chose PEK for their children and families who did not. Children who just 
finished PEK constitute the “treatment” group, and children who are just beginning PEK 
constitute the “no-treatment” comparison group. Again, because children in the two groups 
are different ages, a statistical model is used to estimate the difference in scores between 
the two groups right at the program’s September 1 birthday cutoff date for enrollment. At 
this point, the two groups are essentially the same age, but one has participated in PEK and 
the other has not. To date, the birthday cutoff analysis has been conducted twice: once 
when Cohort 1 was beginning kindergarten (treatment group) and Cohort 2 was just 
beginning PEK (no-treatment group), and once when Cohort 2 was beginning kindergarten 
(treatment group) and Cohort 3 was beginning PEK (no-treatment group). 

We made adjustments in the analyses to account for any differences between the two 
cohorts being compared in their demographic characteristics and when in the fall each 
child was tested. We also made tentative adjustments for differences in baseline scores 
between the cohorts being compared. Even though we made adjustments, it is important 
to note that we have some reservations about the birthday cutoff method based on the 
differences in baseline (fall of PEK) test scores between treatment and comparison 
groups. These differences suggest that assumptions about the equivalency of the two 
groups when they started PEK did not hold in some cases, and it is possible that our 
adjustments may not have entirely corrected for the impact on results (Mueller, 2008). 

Cohort 1 entering kindergarten compared to Cohort 2 entering PEK 

Based on the birthday cutoff analysis, when PEK school-based Cohort 1 children started 
kindergarten they were considerably ahead of same-age children who had chosen but not 
yet received PEK. Again, this is based on statistical estimates of differences between 
Cohort 1 and Cohort 2 at the program’s September 1 birthday cutoff date, where they were 
essentially the same age. Cohort 1 had completed PEK, and Cohort 2 was just beginning 
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the program. There were statistically significant differences in vocabulary, reading, 
writing, and math test scores at the birthday cutoff date in favor of children who had 
attended PEK. The size of the PEK impact on scores is estimated to be between medium 
and large for vocabulary and reading, and large for writing and math (Figure A12). 
However, the size of the program’s impact on writing and math might be overestimated 
due to significant differences between Cohorts 1 and 2 in their pretest (fall of preschool) 
writing and math scores that raise concerns about the equivalency of cohorts at the cutoff. 
Our previous reports included an adjustment for these differences, which estimated that 
the impact of PEK might be medium to large rather than large. We are currently exploring 
better statistical methods for adjusting for these differences to minimize any potential bias. 

In age-equivalency terms, this analysis found a difference of 12 months between the two 
groups in their vocabulary scores. This means that children who attended PEK were 
estimated to be 12 months ahead of where they would have been without attending the 
program. Children who attended PEK were estimated to be eight months ahead in 
reading, 12 months ahead in writing, and 10 months ahead in math compared to where 
they would have been without participating in PEK (Figure A13; Mueller & Gozali-Lee, 
2007). However, differences between the cohorts at baseline suggest that the writing and 
math results may be overestimated. Previous reports included rough estimates adjusting 
for the differences at baseline, which suggest that the children who attended PEK were 
ahead by nine months in writing and by six months in math. We are currently developing 
a more appropriate statistical adjustment. As has been seen in some other analyses of 
PEK results, a look at the impact within individual demographic groups suggests that 
Cohort 1 White students benefited less from the program than other students (Mueller & 
Gozali-Lee, 2007). 

Cohort 2 entering kindergarten compared to Cohort 3 entering PEK 

A year later, we conducted the birthday cutoff analysis when Cohort 2 was entering 
kindergarten and Cohort 3 was entering PEK. In this case, Cohort 2 served as the 
“treatment” group and Cohort 3 as the “no-treatment” comparison group. Again, we used 
a statistical model to estimate differences between the two groups at the program’s 
birthday cutoff point, where the groups were essentially the same age. 

As with the initial analyses based on Cohorts 1 and 2, results again indicated that children 
who had participated in PEK had substantially more advanced skills in vocabulary, reading, 
and writing compared to same-age children who had chosen but not received the 
program, as evidenced by statistically significant differences in test scores at the birthday 
cutoff date. The size of the PEK impact was estimated to be between medium and large 
in these areas. On the other hand, the math advantage observed for Cohort 1 was not 
observed for Cohort 2, as there was not a statistically significant difference in math test 
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score at the birthday cutoff date (Figure A 12). However, the writing and math results are 
likely to be underestimated due to significant differences between Cohorts 2 and 3 in 
their pretest (fall of preschool) writing and math scores. Our previous reports included an 
adjustment for these differences, which estimated that the impact of PEK might be large 
(rather than medium to large) for writing, and small (rather than insignificant) for math. 
Because incoming PEK cohorts differed in their baseline test scores, comparing the 
birthday cutoff results for Cohorts 1 and 2 may be misleading. As noted above, we are 
currently exploring better statistical adjustments to minimize any potential bias resulting 
from the differences at baseline. 

In age-equivalency terms, PEK children were estimated to have a 10-month advantage in 
vocabulary, a 6-month advantage in early reading skills, a 9-month advantage in early 
writing skills, and a 3-month advantage in early math skills (Figure A14; Mueller, 2008). 
Differences between the cohorts in baseline scores suggest that the writing and math 
results may be underestimated. A crude adjustment for these differences suggests that 
PEK children may have experienced a 12-month advantage in writing and a 4-month 
advantage in math. However, we are currently developing a more appropriate statistical 
adjustment to account for the differences at baseline. 

Comparisons to other programs 

The birthday cutoff method has been used in several studies of state-funded preschool 
programs around the country to detennine program effects on children’s test scores when 
they reached kindergarten. Using these studies, we are able to compare PEK’s results 
with those of state-funded preschool programs in eight other states. Overall, the 
estimated effect tended to be larger for PEK on the Peabody Picture Vocabulary Test, and 
no consistent trend on the Woodcock- Johnson sub tests, based on the two birthday cutoff 
analyses conducted thus far (Figures A12 & A15; Mueller & Gozali-Lee, 2007). 

However, there are limitations to these comparisons that should be kept in mind. As 
previously mentioned, we originally made adjustments where there were differences in 
baseline test scores of PEK cohorts being compared, and it is possible that our 
adjustments did not entirely correct for the impact on results. Other studies’ limitations 
in this area are unknown because baseline assessments were not available for both 
cohorts. Additionally, the proportion of English Language Learners in our study may 
account for some of the difference in results, and we will continue to examine the 
implications of the large ELL population as the study progresses. There could also be 
other meaningful differences between the programs. 
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Kindergarten readiness compared to classmates 



When they reach kindergarten, fonner PEK students are also compared to their 
kindergarten classmates. Former PEK school students are followed as long as they attend 
kindergarten in Saint Paul, even if they attend kindergarten at a school other than the 10 
original PEK schools. The classmate comparison group is defined as kindergarten 
classmates of fonner PEK children at the 10 original PEK schools. Some classmates 
have had prior preschool or child care center experience, and some have not. We 
compare former PEK students to each of these two classmate comparison groups: those 
with prior preschool or child care center experience and those without. Kindergarten 
readiness compared to classmates is examined using the Peabody and Woodcock-Johnson 
academic assessments conducted by Wilder Research, as well as the Social Skills Rating 
System assessments completed by teachers. 

Analyses presented here incorporate adjustments for differences among the groups in 
their demographic characteristics and when in the fall children were tested. It is 
important to note that fonner PEK children may also differ from their kindergarten 
classmates in other important ways. For example, families who apply for PEK may differ 
in motivation, knowledge, or other important factors from those who do not. In that 
sense, the birthday cutoff analysis offers advantages. Still, we feel that comparing former 
PEK students to their kindergarten classmates provides insights into how PEK compares 
to other experiences children may have before kindergarten. 

Academic assessments 

In fall of their kindergarten year, children who had participated in PEK scored higher on 
average in each of the four academic areas than kindergarten classmates who had other 
preschool or child care center experience. Classmates without prior preschool or child 
care center experience scored lowest of the three groups on average in each area. The 
academic advantage for PEK children compared to their kindergarten classmates with 
prior preschool or child care center experience was significant in vocabulary only for 
Cohort 1 children, while children in Cohorts 2 and 3 experienced a significant advantage 
in all four academic areas. Compared to classmates without prior preschool experience, 
PEK children in all three cohorts experienced significant advantages on average in all four 
academic areas, including vocabulary, reading, writing, and math (Figures A16-A18). 

Comparing PEK children to classmates who had preschool experience, the effect sizes 
tend to be in or near the small to medium range. The size of PEK’s effects are generally 
larger, tending to be in or near the medium to large range, when PEK children are 
compared to classmates who did not attend preschool or a child care center (Figure A 19). 
These results suggest that PEK provides benefits beyond those received by most 
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kindergarten children in their pre-kindergarten experiences. In addition, the results 
suggest that the size of PEK’s effect has grown with each successive cohort. 

Despite the significant advantages for children who attended PEK, their test scores in fall 
of kindergarten were nonetheless below the national average for vocabulary and math. 

On the other hand, their scores in reading and writing were above the national average. 

Age-equivalency results 

Translating scores into age-equivalency tenns provides another meaningful way to 
examine these results, although it should be noted again that age-equivalency scores are a 
less exact measure. 

PEK children were estimated to have an advantage over classmates with other preschool 
or child care experiences in vocabulary, ranging from a three-month advantage for 
Cohort 1, to a six -month advantage for Cohort 2 and an eight-month advantage for 
Cohort 3. Compared to classmates without prior preschool experience, the PEK 
advantage in vocabulary was estimated to be of 5 months for Cohort 1, 9 months for 
Cohort 2, and 1 1 months for Cohort 3. 

In reading, PEK children in Cohort 1 had similar scores on average compared to their 
classmates with prior preschool experience, while PEK children in Cohorts 2 and 3 were 
estimated to have an advantage. This advantage amounted to five months for Cohort 2 
and two months for Cohort 3. When compared to classmates without prior preschool 
experience, all three PEK cohorts had an advantage in reading. This advantage was 
estimated to be of four months for Cohort 1, six months for Cohort 2, and five months for 
Cohort 3. 

While PEK children in Cohort 1 appeared to have similar writing skills compared to their 
classmates with prior preschool experience, PEK children in Cohorts 2 and 3 were 
estimated to have a two-month advantage over these peers on average. Compared to 
classmates without prior preschool experience, PEK children were estimated to have an 
advantage in writing ranging from three to five to seven months for children in Cohorts 1, 
2, and 3, respectively. 

In math, PEK children in Cohort 1 had similar skills compared to their classmates with 
prior preschool experience, while Cohort 2 had a two-month advantage and Cohort 3 had 
a four-month advantage, on average. The PEK advantage in math over classmates 
without prior preschool experience ranged from four months for Cohort 1 to six months 
for Cohort 2 and eight months for Cohort 3 (Figure A20). 
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The following figure shows the advantages of PEK in age-equivalency terms compared to 
the three comparison groups we have discussed: 1) the birthday cutoff comparison group 
discussed in the previous section, 2) kindergarten classmates without prior preschool or 
child care center experience, and 3) kindergarten classmates with prior preschool or child 
care center experience. As shown in the figure, there tends to be a pattern of stronger 
advantages over kindergarten classmates for Cohort 2 compared to Cohort 1, and stronger 
advantages for Cohort 3 than for both of the previous cohorts. Results of the birthday 
cutoff analysis appear stronger for Cohort 1 than Cohort 2 in general, but again may reflect 
the impact of differences in baseline test scores in writing and math that may overestimate 
the results for Cohort 1 and underestimate the results for Cohort 2. As mentioned, we are 
currently developing a statistical adjustment to correct for these differences. 
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7. PEK school component. Difference in age-equivalency scores in kindergarten: 
PEK students compared to peer groups 



Vocabulary 




Compared to similar peers Compared to classmates Compared to classmates 
(birthday cutoff) without preschool with preschool 



Reading 




Compared to similar peers Compared to classmates Compared to classmates 
(birthday cutoff) without preschool with preschool 



Writing 




Compared to similar peers Compared to classmates Compared to classmates 
(birthday cutoff) without preschool with preschool 



Math 

10 




Compared to similar peers Compared to classmates Compared to classmates 
(birthday cutoff) without preschool with preschool 



■ Cohort 1 □Cohort 2 □ Cohort 3 

Note: This figure presents the differences in months between the average age-equivaiency scores of PEK Cohorts 1, 2, and 3 

and their peer groups upon kindergarten entry, shown only for differences that were statisticaiiy significant based on the standard score 
results. The birthday cutoff analysis (first set of columns) was done for Cohorts 1 and 2 only. Positive numbers indicate that the PEK 
age-equivalency score was higher by that number of months than the peer group age-equivalency score. In other words, children who 
attended PEK were estimated to be that many months ahead of children in the peer group upon kindergarten entry on average. AH 
scores are adjusted for demographic and test date differences between the groups being compared, ns = No significant difference 
between the PEK cohort and the comparison group. The superscript numeral signifies the cohort. 
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Teacher ratings 

Using the Social Skills Rating System, teachers rated fonner PEK children and their 
kindergarten classmates on their social skills, problem behaviors, and academic 
competence in fall of their kindergarten year. The analyses presented here incorporate 
adjustments for differences among the PEK and classmate comparison groups in their 
demographic characteristics. In the area of social skills, former PEK students received 
more positive ratings on average compared to the two kindergarten classmate groups, 
those with and those without preschool experience, which had similar scores in all three 
cohorts. This same pattern was observed in teachers’ ratings of children’s problem 
behaviors for Cohorts 2 and 3, while the groups were similar for Cohort 1. In the area of 
academic competence, former PEK students received the highest ratings on average, 
followed by kindergarten classmates with prior preschool or child care center experience, 
and then by classmates without prior preschool or child care center experience. 

Although PEK children generally had the most positive ratings, the advantages for 
children in Cohort 1 over their kindergarten classmates with preschool experience were 
not statistically significant. In contrast, when compared to classmates without preschool 
experience, children in Cohort 1 did experience a significant advantage on average in the 
areas of social skills and academic competence, but not in problem behaviors. Results 
were stronger for former PEK children in Cohorts 2 and 3. These children had more 
positive teacher ratings on average in each of the three areas than both of the classmate 
groups, those with and those without prior preschool or child care center experience. In 
all three areas, differences between PEK children in Cohorts 2 and 3 and the two 
classmate groups were statistically significant (Figures 8 and A21-A23). 

Compared to national norms, PEK children in all three cohorts exhibited stronger social 
skills and fewer problem behaviors on average. On the other hand, teachers’ ratings of 
their academic competence were below national norms on average. 
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8. PEK school component. Teachers’ ratings in kindergarten: PEK students 
vs. kindergarten classmates 



Assessment 

Social Skills Rating System 


PEK Cohort 1 compared to kindergarten classmates 3 

With preschool/ Without preschool/ 

child care center child care center 


Total Social Skills' 3 


No difference 


Higher for PEK 


Problem Behaviors 0 


No difference 


No difference 


Academic Competence 11 


No difference 


Higher for PEK 




PEK Cohort 2 compared to kindergarten classmates 3 




With preschool/ 
child care center 


Without preschool/ 
child care center 


Total Social Skills' 3 


Higher for PEK 


Higher for PEK 


Problem Behaviors 0 


Lower for PEK 


Lower for PEK 


Academic Competence 11 


Higher for PEK 


Higher for PEK 




PEK Cohort 3 compared to kindergarten classmates 3 




With preschool/ 
child care center 


Without preschool/ 
child care center 


Total Social Skills' 3 


Higher for PEK 


Higher for PEK 


Problem Behaviors 0 


Lower for PEK 


Lower for PEK 


Academic Competence 11 


Higher for PEK 


Higher for PEK 



Note: Includes only students who were tested on both social and academic skills. The analysis adjusted for gender ; age, 

race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 
among the groups being compared. 

a Kindergarten classmates were divided into two groups - those who attended preschool, Head Start or a child care 
center prior to attending kindergarten, and those who did not. 

b Higher scores indicate higher social skills. 

c Higher scores indicate more problem behaviors. 

d Higher scores indicate higher academic competence. 
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Differences in first grade compared to classmates 



Former PEK participants in Cohorts 1 and 2 were compared to their first-grade 
classmates using the same assessments used in earlier years, the Peabody and Woodcock- 
Johnson academic assessments and the Social Skills Rating System. The classmate 
comparison group consists of PEK children’s kindergarten classmates in the 10 PEK 
schools. After kindergarten, students in both the former PEK group and the classmate 
comparison group are followed as long as they remain in schools in Saint Paul. 

Academic assessments 

Progress between kindergarten and first grade 

Between fall of kindergarten and fall of first grade, former PEK students in Cohorts 1 and 
2 made faster progress than their peers nationally on all four academic assessments, 
measuring vocabulary and early reading, writing, and math skills. While gains in all four 
areas were statistically significant for PEK Cohort 1, students in PEK Cohort 2 made 
significant gains in reading and math only, and not in vocabulary and writing. 

Although PEK students made accelerated progress compared to their peers nationally, 
their classmate comparison groups made even more accelerated progress on each of the 
four academic measures during the kindergarten year, on average. In addition, their gains 
were statistically significant in all four academic areas. As shown in Figure 9, 
classmates’ larger gains narrowed the gaps that were seen between former PEK students 
and their classmates in fall of kindergarten. Still, former PEK students continued to score 
higher on average than their classmates in all four academic areas in fall of first grade 
(Figures A24-A25). It should be noted that former PEK students’ progress was compared 
to the total classmate comparison group, including both those with and those without 
prior preschool or child care center experience. 
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Standard scores Standard scores 



9. PEK school component. Changes in academic test standard scores from 

kindergarten to first grade: PEK Cohort 1 vs. kindergarten classmates* (fall 2006 to 
fall 2007) and PEK Cohort 2 vs. kindergarten classmates* (fall 2007 to fall 2008) 

Cohort 1 vs. Kindergarten classmates Cohort 2 vs. Kindergarten classmates 

Vocabulary Vocabulary 




Fall of Kindergarten Fall of 1st grade 



Fall of Kindergarten Fall of 1st grade 



Reading 



Reading 




Fall of Kindergarten 



Fall of 1st grade 



I 1 1 

Fall of Kindergarten Fall of 1st grade 



-♦—PEK Cohort 1 (N=237-8) —♦—PEK Cohort 2 (N=224-225) 

■■—Kindergarten classmates (N=258-61) —■—Kindergarten classmates (N=210-214) 
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Standard scores Standard scores 



9. PEK school component. Changes in academic test standard scores from 

kindergarten to first grade: PEK Cohort 1 vs. kindergarten classmates* (fall 2006 to 
fall 2007) and PEK Cohort 2 vs. kindergarten classmates* (fall 2007 to fall 2008) 
(continued) 

Cohort 1 vs. Kindergarten classmates Cohort 2 vs. Kindergarten classmates 



Writing 



Writing 




104.0 

102.5 




Fall of Kindergarten Fall of 1st grade 

Math 

102.7 
100.4 

90.1 





in 

2? 

O 
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Fall of Kindergarten Fall of 1st grade 

Math 

103.9 
100.2 

90.0 





I 1 1 

Fall of Kindergarten Fall of 1st grade 

—♦—PEK Cohort 1 (N=237-8) 

-■- Kindergarten classmates (N=258-61) 



I 1 1 

Fall of Kindergarten Fall of 1st grade 

—♦—PEK Cohort 2 (N=224-225) 
—■—Kindergarten classmates (N=210-214) 



Notes: Standard scores have a mean of 100 and a standard deviation of 15 in the national normative sample. These 

scores are also age-standardized. This means that no change in scores from one year to the next indicates normative progress , 
positive change indicates accelerated progress , and negative change indicates slower progress in comparison to one’s peers. 
One-year changes in standard scores were statisticaiiy significant for each group in each subject (see Figures A24-A25). 

* The classmate comparison group was defined as kindergarten classmates of former PEK student in the 10 PEK schools. For 

purposes of this analysis , the kindergarten classmate group includes both classmates with and classmates without prior preschool or child 
care experience. 
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The difference in progress between the two groups can also be viewed in terms of age- 
equivalency scores. In the fall of kindergarten, the average age-equivalency vocabulary 
score for PEK Cohort 1 children was 4 years 1 1 months. It increased to 6 years 1 month 
in the fall of first grade, for a gain of 14 months. The comparable age-equivalency scores 
for PEK Cohort 2 children were 5 years 0 months and 6 years 2 months, again 
representing a 14-month gain in vocabulary during the kindergarten year. Nonetheless, 
the classmate comparison groups made even larger gains in vocabulary on average, 
amounting to 15 months for classmates of Cohort 1 and 18 months for classmates of 
Cohort 2. Likewise, the number of months gained is higher for the classmate comparison 
groups than for the PEK students in the areas of reading, writing, and math. Still, the two 
PEK cohorts and their respective classmate comparison groups made faster progress than 
their peers nationally on each of the measures, and all the groups made exceptionally fast 
progress in early math skills (Figures A26-A27). 

PEK’s impact in first grade 

To estimate the potential ongoing impact of PEK, we compared PEK Cohort 1 and 
Cohort 2 students’ fall of first grade test results with those of students in their respective 
classmate comparison groups. As was done in kindergarten, we separated the classmate 
groups into those with other preschool or child care center experience prior to 
kindergarten and those without. Analyses presented here incorporate adjustments for 
demographic differences among PEK and classmate comparison groups as well as when 
in the fall each child was tested. Results suggest that the academic advantages that 
Cohort 1 and Cohort 2 gained from attending PEK had lessened somewhat from fall of 
kindergarten to fall of first grade. 

In fall of first grade, we continued to see advantages over classmates who did not have 
other preschool or child care center experience prior to kindergarten. Cohort 1 and 
Cohort 2 students scored higher on average than their respective classmate comparison 
groups on three of the four measures. The exception was early math skills (Figures A28- 
A29). As shown in Figure 10, Cohort 1 and Cohort 2 students were an estimated four 
months ahead of their classmates in vocabulary. In reading, Cohort 1 had a three-month 
advantage and Cohort 2 had a one-month advantage over their classmates. In addition, 
fonner PEK students were estimated to have an advantage in writing of two months for 
Cohort 1 and one month for Cohort 2. Math scores were not significantly different 
(Figures 10 and A30). 

Compared to classmates who did have other preschool or child care center experience 
before kindergarten, PEK Cohort 1 students did not score significantly differently on any 
of measures in fall of first grade. Results were stronger for students in PEK Cohort 2, 
who maintained significant advantages over their classmates with preschool experience in 
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Months Months 



the areas of reading and writing. These advantages amounted to two months in reading 
and one month in writing. On the other hand, by fall of first grade, Cohort 2 students no 
longer had significant advantages over classmates with preschool experience in the areas 
of vocabulary and math (Figures A28-A29). 



10. PEK school component. Difference in age-equivalency scores in first grade: 
PEK Cohorts 1 and 2 compared to their classmates* 



Vocabulary 



4 4 




Compared to classmates Compared to classmates 
without preschool with preschool 



Reading 

3 




Compared to classmates Compared to classmates 
without preschool with preschool 



Writing 



2 




Compared to classmates Compared to classmates 
without preschool with preschool 



Math 

in 

£ 

+■* 

c 

0 

ns 1 ns 2 ns 1 ns 2 

1 1 

Compared to classmates Compared to classmates 
without preschool with preschool 



■ Cohort 1 □Cohort 2 



Note: This figure presents the differences in months between the age-equivalency scores of PEK Cohorts 1 and 2 and 

their respective classmate comparison groups in fall of first grade , shown only for differences that were statistically significant 
based on the standard score results. Positive numbers indicate that the PEK age-equivalency score was higher by that 
number of months than the classmate group age-equivalency score. In other words , children who attended PEK were 
estimated to be that many months ahead of children in the classmate group when they entered first grade. All scores are 
adjusted for demographic and test date differences between the groups being compared. 

ns = No significant difference between the PEK cohort and the comparison group. The superscript numeral signifies the 
cohort. 

* The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK 
schools. After kindergarten, they are followed as long as they remain in schools in Saint Paul. 
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Teacher ratings 



As was done in kindergarten, teachers used the Social Skills Rating System to rate former 
PEK children and their classmates on their social skills, problem behaviors, and academic 
competence in fall of first grade. The analyses presented here incorporate adjustments 
for demographic differences among the PEK and classmate comparison groups. As was 
observed in the academic assessment results, the social skills results also suggest that 
PEK children’s advantages were reduced by fall of first grade. 

As previously described, in the fall of kindergarten, fonner PEK children in Cohorts 1 
and 2 received significantly higher teacher ratings in social skills on average compared to 
their classmates without preschool or child care center experiences before kindergarten. 
PEK Cohort 2 also had a significant advantage in social skills over their classmates with 
preschool experience. However, a year later, in fall of first grade, these advantages in 
social skills were no longer evident. 

As for problem behaviors, PEK children in Cohort 1 were rated similarly to their 
classmates with and without preschool experience both in fall of kindergarten and in fall 
of first grade. On the other hand, PEK Cohort 2 was rated as exhibiting significantly 
fewer problem behaviors in kindergarten compared to both of their classmate comparison 
groups, those with and those without preschool experience. By fall of first grade, PEK 
Cohort 2 continued to have a significant advantage in behavior over classmates with 
preschool experience, but not over classmates without such experience. 

In academic competence, PEK students in Cohorts 1 and 2 had an advantage in fall of 
kindergarten over their respective classmate comparison groups with no preschool 
experience. In addition, Cohort 2 had an advantage over their classmates with preschool 
experience. By fall of first grade, PEK Cohort 1 maintained its advantage in academic 
competence over classmates without experience. For PEK Cohort 2, the advantage over 
classmates with preschool experience was maintained, while the advantage over 
classmates without experience was no longer evident (Figures 1 1 and A31-A32). 
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11. PEK school component. Teachers’ ratings in first grade: PEK students vs. 
kindergarten classmates 



Assessment 

Social Skills Rating System 


PEK Cohort 1 compared to kindergarten classmates 3 

With preschool/ Without preschool/ 

child care center child care center 


Total Social Skills' 3 


No difference 


No difference 


Problem Behaviors 0 


No difference 


No difference 


Academic Competence 11 


No difference 


Higher for PEK 




PEK Cohort 2 compared to kindergarten classmates 3 




With preschool/ 
child care center 


Without preschool/ 
child care center 


Total Social Skills' 3 


No difference 


No difference 


Problem Behaviors 0 


Lower for PEK 


No difference 


Academic Competence 11 


Higher for PEK 


No difference 



Note: Includes only students who were tested on both social and academic skills. The analysis adjusted for gender , age , 

race/ethnicity, free/reduced-prlce lunch eligibility, English Language Learner status, and Special Education status differences 
among the groups being compared. 

a Kindergarten classmates were divided into two groups - those who attended preschool, Head Start or a child care 
center prior to attending kindergarten, and those who did not. 

b Higher scores indicate higher social skills. 
c Higher scores indicate more problem behaviors. 

d Higher scores indicate higher academic competence. 



Implementation efforts 

This section explores the extent to which PEK’s school component has been implemented 
as intended. Implementation results provide insights into factors that may have contributed 
to the changes seen in Cohort 1, 2, and 3 school participants. The two paragraphs below 
briefly describe the implementation results through the end of the 2007-08 school year. 

For the first three years (2005-06 through 2007-08), the implementation evaluation 
examined the extent to which PEK classrooms align with the district’s Project for 
Academic Excellence and promote language and literacy development. Surveys with 
principals, teachers, and parents were also conducted to assess their satisfaction with 
PEK; teachers’ communication with parents; and parent involvement in children’s 
learning and school activities. Results through the 2007-08 school year indicated that 
program implementation had increased over time. By the end of the third year, all PEK 
classrooms were fully implementing at least a majority of the indicators in the Project for 
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Academic Excellence. PEK classrooms had achieved a high rate of implementation of 
the Early Childhood Workshop model. Classrooms also showed a high rate of 
implementation for most of the indicators related to classroom rituals and routines. 
Classrooms generally met indicators related to classroom environment, although in some 
areas there was room for moving beyond the basic expectations. The evaluation also 
specifically addressed the extent to which classrooms promote literacy and language 
development. To this end, independent observers conduct assessments each year using a 
research-based tool for preschool classrooms, the Early Language Literacy and 
Classroom Observation (ELLCO). Spring 2008 assessments found that overall, PEK 
classrooms created a strong “culture of literacy,” and the impact of PEK’s coaching was 
evident in teachers’ practices. 

Principals, teachers, and parents in 2007-08 reported high satisfaction with the PEK 
program. Principals spoke positively about the leadership and support provided by PEK 
staff. PEK teachers strongly agreed that their school better prepares children for 
kindergarten because of the school’s participation in PEK. Most teachers also strongly 
agreed that participation in PEK professional development has had a large impact on their 
teaching practices. Parents rated their child’s experience in PEK as “excellent” (70% of 
parents) and “very good” (29%). Nearly all parents reported that there is enough effort 
made to involve parents, and parents were very satisfied with PEK teachers’ 
communication. 

Detailed implementation reports were provided by the Saint Paul Public Schools program 
evaluator (Heinrich, 2007) and the independent consultant of Saint Paul Public Schools 
(Passe, 2008), and summarized in our previous reports (Mohr, Mueller, & Gozali-Lee, 
2008b; Mueller & Gozali-Lee, 2007). 

The 2008-09 evaluation follows former PEK children who were in first grade (Cohort 2) 
and in kindergarten (Cohort 3) during that year. Therefore, the implementation 
evaluation focuses on the integration between PEK and kindergarten classrooms and 
professional development offered to kindergarten teachers. 

Implementation findings presented here are organized into the following topics: 

■ Principal and kindergarten teacher satisfaction 

■ Professional development 

■ School integration 
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Principal and kindergarten teacher satisfaction 

Principal satisfaction 

In spring 2009 Wilder Research conducted surveys with principals at the 10 original PEK 
schools. Five principals completed the survey on their own, and the other five principals 
completed the survey as a phone interview with Wilder Research staff. Two of the 10 
principals reported that this was the first year that they were the principal at the school. 
Responding to a series of open-ended questions, the principals reported being satisfied 
with the PEK program. They appreciated having PEK in their schools. According to 
principals, PEK has noticeably improved students’ readiness for kindergarten, and has 
pushed kindergarten classrooms to a higher level. Principals also appreciated building 
the connections with parents early on by having PEK in their schools. 

Following are examples of principals’ comments: 

[The most positive aspects of having PEK classrooms are] that the students are 
prepared for kindergarten, academically and socially, and [they] have an impact 
on the other students who haven’t had the opportunity to be in PEK as role 
models, peer models. [The program resulted in] the kind of readers that are being 
developed, that are showing through observations and assessments. Student 
achievement is greatly increased at the kindergarten level and working on up. 

Also, the program gives the opportunity to develop the relationships with the 
parents at an early stage and to build on that. 

Our PEK students are our role models of the school! PEK really helps to prepare 
the students coming into the school, because they have been exposed to the 
rituals and routines of what the school is like. 

PEK provides students with a prior knowledge of the school environment before 
coming into kindergarten and getting students and families engaged early. We 
have a family outreach staff who gives orientations to parents surrounding the 
PEK and kindergarten curriculum, answering questions, such as “What can you 
do to help your child be more successful at school?” and “What can you expect 
[to occur in the classrooms]?” 

Kindergarten students who have entered our classes after a year in PEK are miles 
ahead of other students starting kindergarten. We are constantly amazed with 
their performance. 

The academic rigor that comes out of the program and the exceptional 
professional development that the teachers get from PEK are the positive aspects 
of the program. The most important thing is the success that we are seeing with 
the kids. The popularity of the program as a result of that success makes the 
parents very happy and excited! 
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It has made a tremendous difference in our students’ ability to produce in the 
kindergarten and first-grade classrooms, and it brings in the parents at an early 
time to understand their crucial role in working with their children and dealing 
with their children’s education. 

[The program has contributed to] increased enrollment [in my school] and the 
ability to know families when their children are younger. 

The program provides a positive start for kindergarten for so many students who 
otherwise would miss out. It also provides parent education to parents at an 
earlier stage than we would otherwise be able to do. We are able to demonstrate 
to teachers of K-6 how learning occurs at such an early stage with JOY and to 
demonstrate that rigor in an academic program does not have to equate with a 
disciplinarian approach in order to achieve academic success; rather, it has more 
to do with a systematic approach to how children learn at certain ages. The 
program helps establish relationships with families from an earlier stage and 
maintain those relationships for a longer time throughout the elementary school 
career of their children. 

Principals also spoke favorably of the leadership and support provided by PEK and the 
Saint Paul Public Schools. 

I think the leadership of this program under the direction of the PEK assistant 
director is exceptional. I would like to see that her role be strengthened as we 
look to step up the rigor in our kindergarten and subsequently first-grade 
classrooms. 

Once again I can say without hesitation, PEK is one of the best things Saint Paul 
Public Schools has ever done in my time with the District (12 years). The 
program is superbly organized, continuously improving to meet the changing 
needs of all stakeholders. It is influencing the entire system from the lowest 
grade up. It is doing it in such a way that people are noticing and wanting to 
change rather than feeling that they have to change because someone is making 
them change. 

It is a wonderful program that meets the needs of students on a daily basis. It is 
wonderfully organized from a central administrative perspective, with the PEK 
assistant director and her team [coordinating the program]. The staff 
development has been top notch. And the vision for what is happening has been 
clearly articulated. . . .the communication that they send out is wonderful! They 
are always sending out updates and things to look for. 

Asked specifically about what they would do if they could change some things about 
PEK in their school, principals most frequently described wanting to expand the program. 
They described wishing they had the resources to open an additional classroom, to have a 
bigger classroom space, or wanting longer classroom hours, for example. 
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While principals voiced support for PEK overall, a few of them also suggested ways the 
program could be strengthened. For example, one principal indicated that PEK classrooms in 
her school needs to have a staff member working with the ELL students. Another principal 
suggested flexibility in sharing the PEK staff with the kindergarten classrooms. She 
mentioned that, in her school, the PEK staff help with guided reading and instruction in 
kindergarten classrooms, while the kindergarten ELL staff work with the PEK children in 
active learning. One principal felt that she needs to encourage better connections between 
PEK and kindergarten teachers. Another principal said better aligmnent between PEK and 
kindergarten curriculum and practices is needed in her school. Additional parent information 
sessions and bussing to the neighborhood schools were also suggested by one principal. 

Teacher satisfaction 

In spring 2009 Wilder Research asked the kindergarten teachers in the 10 original PEK 
schools to complete a survey about their experiences with the program. Twenty-five of 
the 26 eligible teachers completed the survey, including 23 teachers who completed the 
survey on their own and 2 teachers who completed the survey as a phone interview with 
Wilder Research staff. One teacher was on leave when the survey was conducted and the 
substitute teacher was unfamiliar with the program. Results indicate that all teachers 
agreed, indicating “strongly agree” (92%) and “somewhat agree” (8%), that their school 
better prepares children for kindergarten because of the school’s participation in PEK. 

Most teachers described how the PEK children are more prepared academically and 
socially to enter kindergarten, as compared to other children. Examples of teachers’ 
comments follow: 

The children who participated in Pre-K are coming to school knowing the basics 
of being in school. I believe this is very beneficial. They are also coming to 
kindergarten with stronger pre-literacy skills. 

[The children] come to school prepared and know what school is about. Their 
behavior and reading, writing, and math skills are higher than children who have 
not participated in the Pre-K experience. 

Students are more comfortable and confident about all aspects of school when 
they arrive. Most students who come from Pre-K have higher achievement and 
learn how to help other students. 

The students are better prepared academically and socially. The families are 
already accustomed to school routines and schedules. 
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Because of the existence of the Pre-K program, my incoming students come well 
prepared for school. Learners coming from this program know the essential 
concepts necessary to begin kindergarten. Students from the Pre-K program 
enter my classroom knowing how to write their name, identify letters and 
numbers, hold a pencil, cut with scissors, etc. 1 see a wide learning gap between 
the children who attended the Pre-K program vs. those who did not. 

Pre-K has done a wonderful job getting kids ready to read. Many come in 
knowing their letters and concepts about print. Math skills are also stronger. 



Professional development 

PEK has made a number of strides in fostering linkages between PEK and kindergarten 
teachers. Such linkages are necessary to ensure children are well prepared for 
kindergarten, and to ensure their gains in PEK are built on and sustained in subsequent 
years. Understanding the skills of incoming PEK students can also help kindergarten 
teachers prepare to differentiate their instruction. Fostering these linkages is an ongoing 
process, and the program continues to focus attention in this area. 

School integration 

Principals’ perceptions 

In the 2009 survey, principals were asked if they had seen improved connections between 
PEK and kindergarten classrooms in their school. The principals in the four schools that 
received intensive professional development from PEK (Dayton’s Bluff, Wellstone, 
American Indian, and World Cultures) described seeing the program coach work closely 
with both the PEK and kindergarten teachers in Professional Learning Communities on a 
regular basis. Principals said that teachers work together to discuss instructional 
strategies and visit each other’s classrooms. 

Their comments follow: 

Absolutely! The PEK coach [name of person] works with the PEK and 
kindergarten teachers in Professional Learning Communities (PLC). They met 
regularly and addressed goals to improve instruction that was visible and has 
shown good results [for our students] in the district’s reading assessment, PALS 
for kindergarten. 

Most definitely. Our PEK and kindergarten teachers have been meeting as 
Professional Learning Communities this year, which has included a video sharing 
of instructional practices at each meeting. Together these teachers have visited 
each other’s classroom and shared ideas. I see a much stronger alignment with 
the instructional strategies and specifically in oral language development. 
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I have observed a more defined vision and focus surrounding the program. I 
have also seen a stronger alignment surrounding what the 4-year-old program 
does in relation to preparing students for kindergarten. There has been more of 
an aggressive attempt to align instructionally and make those kinds of 
connections between PEK and kindergarten classrooms. The PEK staff are doing 
more and more each year. The program is becoming more and more defined. 

The PLC group has been meeting on a regular basis, and there is a lot of “aha” 
moments (“1 can do it!” “1 can see how that would go!”). My PEK teacher went 
to a kindergarten classroom and [observed] the things that they were doing. 
[However], one of the things that didn’t happen this year is having my 
kindergarten classrooms to be more aligned with PEK. We need to get my 
kindergarten teachers to look into the PEK classrooms, because there is much 
more rigor in PEK. 



To a lesser degree, other school principals also mentioned some connections between the 
PEK and kindergarten classrooms in their schools. Most often, they indicated their 
teachers received professional development from PEK and the district, and that all 
teachers participated in learning walk s . A few principals also mentioned that the PEK 
and kindergarten teachers in their schools share information in Professional Learning 
Communities. However, one principal noted that the instructional models in those 
classrooms are different. 

Asked about ideas for furthering the connections between PEK and the rest of the 
schools, many principals mentioned that they want to expand the Professional Learning 
Communities to include first grade classrooms, continue the joint professional 
development, and have learning walks throughout the school. One principal also wanted 
to have more time for teachers to collaborate and use the same assessments in 
kindergarten classrooms as in the PEK classrooms to measure student progress. Another 
principal said that she plans to have the PEK children who are reading above grade level 
participate in the Family Reading Everyday program alongside the kindergarten children 
in her school. However, a principal voiced her concern about challenges to differentiate 
and to offer the same supports in kindergarten as in the PEK classrooms because of the 
lower number of staff available in kindergarten. 

Principals also described the importance of sharing information and providing support to 
each other in their role as the instructional leader of PEK at their schools. 

Teachers’ perceptions 

The spring 2009 kindergarten teacher survey also included several questions asking 
teachers about the connections with PEK teachers and about differentiated instruction in 
their classrooms. Results in Figure A33 show that all kindergarten teachers agreed 
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(indicating “strongly agree” and “somewhat agree”) that they work with PEK teachers in 
Professional Learning Communities, although more teachers from the four schools who 
received coaching from PEK responded strongly agree (75% vs. 54%) to the statement 
than teachers from the other schools. Almost all teachers also strongly (44%) or 
somewhat (52%) agreed that they have received training and support on how to 
differentiate instruction to meet the needs of their students. 

While kindergarten teachers felt that there are positive connections with PEK teachers, 
information about students’ needs and skills for the purpose of differentiation of 
instruction in kindergarten is not always shared or discussed. Most teachers reported that 
there is sufficient communication between the PEK teachers and kindergarten teachers, 
with 52 percent of the teachers indicating strongly agree and 32 percent somewhat agree. 
Slightly fewer teachers (76%), however, strongly and somewhat agreed that there is 
communication between them and PEK teachers during the kindergarten transition period 
about students’ skills and needs by discussing their assessment results, for example. 

Similarly, while almost all kindergarten teachers (96%) reported using the individual 
student data regularly to inform their teaching, fewer teachers (72%) reported using the 
student information provided by PEK teachers to help develop lessons, activities, and/or 
grouping strategies in the classrooms. 

Asked specifically about observing each other’s classrooms, fewer than half of the 
kindergarten teachers reported that they had observed the PEK classrooms (42%) and had 
their classrooms observed by the PEK teacher (44%). Compared to teachers from the 
other schools, more kindergarten teachers at Dayton’s Bluff, Wellstone, American Indian, 
and World Cultures who received coaching from PEK reported that they had observed 
the PEK classrooms (70% vs. 21%) and had their classroom observed by the PEK 
teachers (73% vs. 21%). 

The spring 2009 kindergarten teacher survey also included an open-ended question 
asking teachers for their ideas for furthering the connections between PEK and 
kindergarten classrooms at their school. Although the degree of collaboration seems to 
vary by school, most teachers suggested ways to further collaboration at their school. 
Their ideas included continuing to learn from the coach and working together with the 
PEK teachers in the Professional Learning Communities (PLCs), more communication, 
more consistent curricula, mutual observations, and shared training experiences. A few 
kindergarten teachers also wished to have more staff and resources in their classrooms. 
Examples of their comments follow: 
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I would love to see observation days built into the yearly schedule; days for the 
Pre-K teacher to observe in kindergarten and days for the kindergarten teachers 
to observe in Pre-K. As a kindergarten teacher, it would be helpful for me to see 
what is happening in the Pre-K classroom so that 1 can extend the instruction in 
my own classroom. 1 would also love to be trained in some of the original Pre-K 
methods. In other words, it would be nice to see more similarities between the 
Pre-K/Kindergarten programs. In order for this to happen, 1 believe the 
kindergarten classrooms would also need similar adult support (i.e., teaching 
assistants to assist students while small group instruction is taking place and 
coaches to help guide instruction). 

I will continue having the team level meeting together like the PLC learning and 
setting goals. Also, [it would be important to] continue having the Pre-K 
coaching person to go around and observe our teaching and also give us 
feedback. 

The classrooms are not in close proximity physically to each other. Informal 
communication doesn't happen easily, and formal collaboration has to be 
carefully planned for. 

1 would like to be able to keep more of the Pre-K families in our building for 
kindergarten. 

[1 would like to have] a chance to share or talk about what the Pre-K teachers are 
teaching, what kind of assessments they are using and just to hear names of kids 
who are doing well and who are struggling. 

[1 would like to have ] more time and opportunity for Pre-K and K teachers to sit 
down when school starts and discuss students. We just don't have time. Maybe 
principals should allow time for that. We are always in group meetings that 1 
don't find as beneficial as 1 would if 1 could talk with and spend time with the 
Pre-K teacher and discuss each student, even in that first week. 

Sharing ideas about common themes and knowing what areas have been taught in 
PEK is needed, so that kindergarten teachers can review, re-teach and add. 
Teaching the same type of material can be helpful if everyone works together. 

It would be better if the classroom was closer to the kindergarten classrooms and 
if we had more time to collaborate about what the students' needs are. Also, it 
would be wonderful to be able to observe each other. 

We are continuing to learn together through coaching and a Pre-K-K PLC group. 

It is important to allow time during staff development for the Pre-K and K 
teachers to collaborate instead of expecting them to do it on their own time. Set 
up Pre-K classrooms so that the teachers and students can interact throughout the 
school day. 
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Kindergarten teachers need to be given the training that Pre-K teachers receive to 
better bridge the learning for students. 

1 think Pre-K came in with their own agenda and curriculum and did not consider 
our curriculum. They started doing units of study that we had been doing for 
many years. I think some consideration should have been given to what was 
already in place. For further connection: Making sure we're not duplicating 
units, and making sure things are developmentally appropriate for both levels. 



Issues for consideration 

A core component of PEK is the inclusion of a rigorous, ongoing evaluation that can be 
used to infonn programming and ultimately assess program results. Based on the findings 
presented in this report, following are several issues that can be taken into consideration 
in future planning for PEK school sites. Some of the issues pertain to PEK staff and 
some pertain to the researchers studying PEK. 

■ Continue to promote differentiating instruction . The success of PEK in increasing the 
skills of participants can result in larger skill differences between them and their 
classmates when they reach kindergarten. These differences pose an instructional 
challenge for kindergarten teachers. For the program and district, they raise 
considerations about how to ensure that all children are able to build on their current 
skills and achieve substantial advances in kindergarten. It is possible that the 
narrowing of differences that we observed between PEK Cohorts 1 and 2 children and 
their classmates from kindergarten to first grade could reflect instruction being targeted 
to a lower skill level than that of fonner PEK students. 

District efforts to expand the PEK model to 4-year-old programs district-wide should 
help address the issue to some extent by increasing the proportion of children who 
enter Saint Paul schools with similar preparation. Still, there will continue to be 
diversity in preparatory experiences among children entering kindergarten. For 
example, most kindergarten teachers in the spring 2009 survey felt that the fonner 
PEK children are more prepared academically and socially to enter kindergarten than 
their classmates. This points to the need for kindergarten teachers to differentiate 
their instruction to the varying skill levels of the children in their class. This is 
important for PEK children so that they maintain and continue to build on the benefits 
that PEK provided, and is also important for children without strong academic 
preparation so they are taught at an appropriate level. Although research on the 
effectiveness of differentiated instruction is still at an early stage, the principles on 
which it is based have some grounding in research (Hall, 2002). The district began 
efforts in this area in 2008-09 as mentioned. 
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■ Collaboration with kindergarten teachers. PEK has also made a number of strides in 
fostering linkages between PEK and kindergarten teachers. Such linkages are 
necessary to ensure children are well prepared for kindergarten, and to ensure their 
gains in PEK are built on and sustained in subsequent years. Understanding the skills 
of incoming PEK students may also help kindergarten teachers prepare to differentiate 
their instruction. Fostering these linkages is an ongoing process, and the program can 
continue to focus attention in this area. The fall 2008 joint training between PEK and 
kindergarten teachers is one important milestone, as is the PEK coaching pilot for 
kindergarten teachers in four schools. Principals’ and kindergarten teachers’ 
enthusiasm for forging these linkages provide a strong basis for continuing to connect 
PEK teachers with kindergarten teachers and the larger school. 

■ Using information provided by PEK teachers to guide instructions in kindergarten 
classrooms. While most kindergarten teachers felt that there is sufficient 
communication between them and the PEK teacher at their school, infonnation about 
students’ needs and skills for the purpose of differentiation of instruction in 
kindergarten is not always discussed or used. More time to observe PEK classrooms, 
share teaching strategies with the PEK teachers, and discuss individual students’ 
strengths and challenges are needed, according to kindergarten teachers. At the same 
time, kindergarten teachers also need to use of the information shared by PEK 
teachers to help develop lessons, activities, and grouping strategies in kindergarten 
classrooms. 

■ Assessing impacts of the intensive coaching provided to some kindergarten teachers. 
As mentioned above, kindergarten teachers in four schools received weekly one-on- 
one coaching from PEK during the 2008-09 school year. They also received 
intensive training on differentiated instruction. To assess whether this effort 
potentially impacts children’s academic skills, we will compare changes in children’s 
academic progress from fall of kindergarten to fall of first grade between those whose 
teachers received coaching and those whose teachers did not. 

■ Using same assessments in Pre-kindergarten as in kindergarten classrooms. In 2008- 
09, the PEK teachers began administering subtests of the Phonological Awareness 
Literacy Screening (PALS). The instrument is also being used in the district’s 
kindergarten classrooms. Using the same assessments in PEK as in kindergarten 
classrooms may help teachers across the grade levels discuss and address individual 
children’s strengths and challenges. Since the district gathers classroom infonnation 
on alignment with the Project for Academic Excellence (PAE), it might be useful to 
review the results with both PEK and kindergarten teachers to address practices that 
can be strengthened at both grade levels, as well as to identify practices that have 
been successfully implemented in PEK classrooms and build on them in kindergarten 
classrooms. 
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■ Additional staff and supports. One principal indicated that their PEK students would 
benefit from having a staff member working with the ELL students. Another 
principal voiced her concern over the challenges to differentiate instruction and to 
offer the same supports in kindergarten as in PEK classrooms because of the lower 
number of staff available in kindergarten. A third principal suggested flexibility in 
sharing staff across the grade levels. PEK staff may want to consider whether there 
are ways of providing more targeted support to ELL students in PEK and possibly 
explore whether there are opportunities to share staff across classrooms. 

■ Building connections with parents. Several principals mentioned that PEK helps the 
schools build connections with parents early on. PEK helps instill parent 
involvement in their children’s education and its success which, in turn, contributes to 
parents wanting to enroll their child’s in the elementary school. Program staff should 
be commended for their efforts to involve parents. 
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Progress summary: Community-based PEK 



This section provides results for the community-based child care portion of PEK. As 
described earlier, PEK extended the program to Saint Paul child care settings in 
recognition that parents use a variety of care arrangements for their children. The 
program considers this component a pilot, with the intent that a community-wide 
approach will help more children enter school with the skills needed to succeed. 
Participating sites include child care centers as well as family child care homes. 

The first group of providers recruited for the program offered PEK from fall 2006 to 
spring 2008, although there was considerable turnover among center teachers and home 
providers during that time. Using what it learned with this initial group of providers, 

PEK launched the program with a second cohort of providers in fall 2008. During 2008- 
09, 7 new centers, 1 continuing center, and 13 new homes offer PEK. It should be noted 
that both child care center teachers and family child care home providers are referred to 
here as “teachers.” As of fall 2009, all 8 child care centers and 10 family child care 
homes are still offering PEK. Three new homes will also join the second cohort of 
community child care providers. 

This section begins by profiling children who participated during the program’s first three 
years in child care settings, 2006-07 (Cohort 1), 2007-08 (Cohort 2), and 2008-09 
(Cohort 3). Their progress during PEK is then discussed based on Individual Growth and 
Development Indicators (IGDIs) administered by PEK staff. Academic and social 
outcomes based on Wilder Research’s assessments are then provided for the first cohort 
of child care children in fall 2007 and second cohort in fall 2008 when they reached 
kindergarten. Upon reaching kindergarten, these children were compared to children 
who had attended PEK at school sites as well as to the same comparison group of 
kindergarten classmates. After summarizing student results, this section describes the 
program’s implementation in child care settings these first three years. The section 
concludes with a list of issues for consideration that can be used to inform future 
planning in the child care component. 

Topics addressed in this section include the following: 

■ Overview of results 

■ Characteristics of children (Cohorts 1-3) 

■ Progress while in PEK (Cohorts 1-3) 

■ Kindergarten readiness compared to classmates (Cohorts 1 and 2) 
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■ Implementation efforts (Cohorts 1-3) 

■ Issues for consideration 

Overview 

Interpreting child care results 

In interpreting results for community-based PEK, there are a couple of issues that should 
be kept in mind. First, it is important to recognize that outcomes available to date for 
PEK child care children are more suggestive than conclusive. This is in part based on the 
small size of the PEK child care group. Additionally, there was a rather large difference 
between the child care group and the comparison groups in the proportion of ELL 
children. Even though we adjusted for demographic differences among the groups, it is 
possible that our adjustments did not entirely correct for the impact of these differences. 

Results should also be viewed in the context of the teacher turnover that is often seen in 
child care settings. Participating child care centers in the first two years experienced high 
teacher turnover. Additionally, several family child care home providers became 
ineligible for the program when changes in their enrollment brought them below the 
program’s minimum enrollment requirements. Changes in family child care home 
providers also resulted from providers leaving the child care field, loosing their child care 
license, or choosing not to continue with PEK. When family child care homes exited the 
program, new providers were asked to take their place for the remainder of the initial 
cohort. A more stable group of teachers participate in the second cohort of community 
child care providers. As of September 2009, the second cohort of providers had been 
offering PEK for one year and most providers will continue with PEK in 2009-10. 

Key findings 

Preliminary results suggest children who participated in PEK’s first two years at child care 
sites experienced some advantages over classmates in kindergarten, but did not perform as 
well as children who attended PEK at school sites. Additional data are needed for 
researchers to make stronger claims about the child care component’s impacts. Over the 
next few years, we also hope to assess differences in results between home and center sites. 
On average, findings for 4-year-olds in the first two child care cohorts were as follows: 

■ When they reached kindergarten, PEK child care Cohorts 1 and 2 children appeared 
to have an advantage over classmates who did not participate in PEK on some 
academic measures, especially in vocabulary. 
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■ However, PEK school-based children appeared to have a slight advantage over PEK 
child care children on reading and math when both groups reached kindergarten. 
Average vocabulary scores were about the same across the child care and school- 
based cohorts. 

■ In the areas of social skills, PEK child care Cohorts 1 and 2 children did not appear to 
have any advantages compared to kindergarten classmates. Based on teachers’ 
ratings, PEK school children exhibited fewer problem behaviors. Classmates with 
other preschool or child care center experiences also appeared to have fewer problem 
behaviors on average. 

■ In the area of academic competence, teacher ratings indicated that PEK child care 
Cohort 1 and 2 children had advantages over classmates without prior preschool or 
child care center experiences. No significant differences in academic competence 
were found between children who attended PEK at school sites and at child care sites. 

Key child care component findings to date also include the following: 

■ Overall, child care teachers participating in focus groups provided positive feedback 
about their experiences with PEK, the helpfulness of PEK’ s professional 
development, and the program’s impact on children. 

■ Almost all parents with children entering kindergarten in the fall said their PEK child 
care teacher helped prepare their child for kindergarten. 

■ Overall, structured classroom observations found that PEK child care sites were 
strong in their support for language and literacy. 

Characteristics of children 

In fall 2006, PEK extended the program to children at participating child care sites in 
Saint Paul. Figure 12 shows the number of children who participated in the first three 
cohorts at PEK child care sites. It is important to note that these data reflect all children 
enrolled in PEK child care during this time, whereas school cohorts are defined as 
students tested in fall of their PEK year. A total of 137 3- and 4-year-old children 
participated in PEK at child care sites during 2006-07 (Cohort 1), 114 participated in 
2007-08 (Cohort 2), and 183 participated in 2008-09 (Cohort 3). Some of those children 
did not participate in PEK for the entire year either because of their entry or exit from the 
child care site or their provider’s entry or exit from the program during the year. Child 
care programs also extend PEK to 2 l / / 2-ycar-olds, although those children are not reported 
on here. 
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12. Children attending PEK child care sites, 2006-07, 2007-08, and 2008-09 



Cohort 


3-year-olds 


4-year-olds* 


Total 


Cohort 1 (PEK 2006-07) 


65 


72 


137 


Cohort 2 (PEK 2007-08) 


59 


55 


114 


Cohort 3 (PEK 2008-09) 


84 


99 


183 



* Some children who participated in Cohort 1 as 3-year-olds also participated in Cohort 2 as 4-year-olds. Similarly, some 
children (n-13) who participated in Cohort 2 as 3-year-olds also participated in Cohort 3 as 4-year-olds. 

Note: Child care Cohorts 1, 2, and 3 reflect all children attending PEK child care. In contrast, school-based cohorts are 

defined as PEK students who were assessed in fall of their PEK year. It should also be noted that child care settings extend 
the program to 2 V 2 -year-olds, although those children are not reported on here. 



Demographics 

In both family child care homes and child care centers, approximately half of the PEK 
participants were age 3 and half age 4 in each of the three years of PEK. Across the 
years, 40-72 percent of the children in family child care homes and 73-91 percent of the 
children in child care centers were in the PEK target population, meaning they were 
English Language Learners, came from low-income families, or needed Special Education 
services. Higher percentages of center than home care children came from low-income 
backgrounds (69-92% vs. 27-53%). PEK child care children typically spoke English as 
their primary home language, including 81 to 100 percent of home care children and 85 to 
94 percent of center care children across the three years. Two to seven children received 
Special Education services each year (Figures A34-A36). Figure 13 shows the 
percentages of PEK child care children in the program’s target populations during the 
first three years in child care settings. 



Project Early Kindergarten evaluation: 
Results through 2008-09 



58 



Wilder Research, September 2009 





13. PEK community component. Representation of PEK target populations, 2006-07 to 2008-09 




Note: PEK targets children who are English Language Learners (ELL), from low-income families, or need Special Education services. “Target 

population” reflects the percentage of children who are in any of these three groups. 



Comparison group demographics 

When they reach kindergarten, PEK child care participants are compared to children who 
participated in the PEK school component as well as children in the school component’s 
comparison group. As in the school component, the comparison group is broken down into 
those with prior preschool or child care center experience and those without. In both 
Cohorts 1 and 2, we found that PEK child care children differed somewhat demographically 
from their kindergarten comparison groups, which included PEK school-based children and 
the comparison group with preschool experience and the one without it. First, the 
proportions of ELL children in these three groups of kindergarten classmates across the two 
cohorts (44-53%) were much higher that in child care Cohort 1 (23%) and Cohort 2 (3%). 
Second, these groups had higher proportions of Asian children (22-38%) than child care 
Cohort 1 (6%) and Cohort 2 (3%). As with analyses in the school component, in cases 
where former PEK child care children differed from comparison group children based on 
demographic characteristics or when in the fall they were tested, we statistically adjusted for 
those differences in our analysis (see Mueller, 2008). 
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Changes over time 

Also as in the school component, it is possible for child care children’s demographic 
characteristics to change over time. For example, some parents may not initially know 
whether their children need Special Education services. As another example, some parents 
may not initially know that their child is eligible for free or reduced-price lunch, may not 
apply until their child enters school, or may experience a change in their eligibility. 

Changes due to attrition 

Following PEK, Wilder Research assesses participants in the community-based portion if 
they attend kindergarten in Saint Paul. As in the school component, children attending 
kindergarten outside of Saint Paul are not reflected in the results. In fall 2007, we were 
able to assess 47 (65%) of the 4-year-olds who had participated in PEK at child care sites 
during 2006-07 (Cohortl) and were beginning in fall 2007. In fall 2008, 34 (62%) of the 
4-year-olds who had participated in PEK at child care sites during 2007-08 (Cohort 2) 
were assessed. 

Attendance 

For children participating in PEK child care Cohort 1, attendance data are available from 
September 1, 2006, through August 31, 2007 (Figure A3 7). For child care Cohort 2, 
attendance data are available from September 1, 2007, through April 30, 2008 (Figure 
A3 8). The initial group of child care providers participating in the program ended their 
contracts with PEK in spring 2008, and complete attendance data were not available for 
the remainder of the year. Attendance data for Cohort 3 children who began with the 
second cohort of providers are available from September 1, 2008 through August 31, 

2009 (Figure A39). 

The number of days children attended during the first two years varied widely, in part 
because some of the family child care homes did not participate in PEK during the entire 
period. From September 2006 through August 2007, 4-year-olds attended an average of 
163 days at family child care homes with a range of 1 1 1-235 days, and attended an 
average of 165 days at child care centers with a range of 38-248 days. Eight (14%) of the 
center children attended 100 or fewer days. Three-year-olds’ attendance was slightly 
higher during that time on average, with an average of 182 days at homes and 168 days at 
centers (Figure A3 7). 

Again, for Cohort 2 attendance data are available for only September 2007 through April 
2008. During these eight months, 4-year-olds attended an average of 134 days at homes 
with a range of 70-158 days, and an average of 122 days at centers with a range of 20- 
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164 days. Four of these home children (22%) and nine of these center children (24%) 
attended 100 or fewer days. Three-year-olds attended an average of 125 days at homes 
and 1 14 days at centers during this time (Figure A38). 

Rate of attendance for Cohort 3 children is generally higher than the previous two 
cohorts. From September 2008 through August 2009, 4-year-olds attended an average of 
181 days at family child care homes with a range of 132 to 216 days, and attended an 
average of 192 days at child care centers with a range of 78 to 249 days. Three-year-olds 
attended an average of 159 days at homes and 198 days at centers (Figure A3 9). 



Progress while in PEK 

Teachers’ assessments of early language and literacy development 

Teachers use Individual Growth and Development Indicators (IGDIs) to monitor 
individual children’s early language and literacy development over time. Preschool 
IGDIs measure children’s progress in three areas: picture naming, alliteration, and 
rhyming. During the assessments, teachers hold up cards with pictures and ask children 
to name pictures, identify pictures starting with the same initial sound, and identify 
pictures that rhyme. The assessments provide teachers with feedback on individual 
children’s progress over time toward developmental outcomes, and alert teachers when 
additional interventions may be needed (ECRIMGD, 1998; Get It! Got It! Go! website, 
n.d.). This section summarizes results for 4-year-olds in PEK. It should be noted, 
however, that IGDIs are also administered to 3 -year-olds in PEK’s community child care 
component. PEK established target scores of 26 for picture naming, 12 for rhyming, and 
8 for alliteration for the end of the pre-kindergarten year. 

PEK child care staff administer IGDIs three times each year. During 2008-09, IGDIs was 
administered in October, January, and April. Results reported here reflect 4-year-olds 
who took the pre-test in October and post-test in April. Similar to the previous cohorts, 
Cohort 3 children improved on all three indicators. Similar to Cohort 2, the biggest 
improvement was in rhyming, while the biggest improvement for Cohort 1 was in picture 
naming. Cohort 2 children scored higher than Cohort 1 and 3 students at their post-test 
(Figure 14). Cohort 2 scores were also higher than Cohorts 1 and 3 at pre-test, and their 
average gains across the three indicators were slightly higher than that of Cohort 1 and 3 
students. Thirty percent of Cohort 3 4-year-olds met the program’s target for picture 
naming at pre-test, and 46 percent at post-test. For rhyming, 19 percent of 4-year-olds 
met the target at pre-test, and 42 percent at post-test. Sixteen percent of 4-year-olds met 
the target for alliteration at pre-test and 36 percent at post-test. Comparison between 
settings show that more children at homes than at center sites met the targets in picture 
naming (50% vs. 44%), rhyming (65% vs. 37%), and alliteration (60% vs. 30%) at post- 
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test. Results should be interpreted with caution, however, because the small number of 
children in each setting. Detailed results for Cohorts 1 and 2 are presented in our 
previous report (Mohr, Gozali-Lee, & Mueller, 2008b). 



14. PEK community component. Percentages of Cohort 1, 2 and 3 children 
meeting IGDI targets at post-test, 2006-07, 2007-08, 2008-09 

85% 




Cohort 1 Cohort 2 Cohort 3 

(N=55-57) (N=38-39) (N=99) 

■ Picture naming Rhyming □ Alliteration 



Kindergarten readiness compared to classmates 

Academic assessments 

When they reach kindergarten, children who participated in PEK at child care centers or 
family child care homes are compared to their kindergarten classmates using the same 
assessments used in the school component, the Peabody and Woodcock- Johnson academic 
assessments and the Social Skills Rating System. First, PEK child care participants are 
compared to the same classmate comparison group as is used in the school component. 

As in the school component, the classmate comparison group is broken down into those 
with other preschool or child care center experiences (other than PEK) and those without 
such experiences. Second, children who attended PEK at child care sites are compared to 
children who attended PEK at school sites. Again, we adjust for demographic and test 
date differences among the groups being compared. It should be noted that these results 
are more suggestive than conclusive due to the small size of the PEK child care groups 
and differences between the groups in the proportion of ELL children. 
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Comparisons to classmate comparison group 

When they reached kindergarten, it appeared that children who were in PEK child care 
had an advantage on some measures compared to their kindergarten classmates who did 
not participate in PEK. However, the advantages were not very consistent across the two 
cohorts. The most consistent finding was for vocabulary, where PEK child care children 
in both Cohorts 1 and 2 scored significantly higher on average compared to classmates 
without prior preschool or child care center experience. PEK child care Cohort 1 also 
scored significantly higher in vocabulary than their classmates with preschool experience. 
In the area of early reading, PEK child care Cohort 1 children scored significantly higher 
on average than the group without preschool or child care center experience but not the 
group with such experience. Children in PEK child care Cohort 2 did not appear to have 
an advantage in early reading over either of the kindergarten classmate comparison 
groups. On the other hand, PEK child care Cohort 2 did have significant advantages on 
average over their classmates without preschool experience, but not over their classmates 
with experience, in the areas of early writing and math skills. However, these advantages 
were not observed for PEK child care Cohort 1 children, who did not significantly differ 
from their classmates in these areas (Figures A40-A41). 

Comparisons to school-based PEK 

PEK child care Cohort 1 was also compared to PEK school-based Cohort 2 when both 
groups reached kindergarten in fall 2007. Likewise, PEK child care Cohort 2 was 
compared to PEK school-based Cohort 3 when both groups reached kindergarten in fall 
2008. Children who attended PEK at school sites scored somewhat higher on average in 
reading and math, but none of the differences were statistically significant. In addition, 
children who attended school-based PEK scored somewhat, though not significantly, 
higher on average in writing compared to child care Cohort 1 but not compared to child 
care Cohort 2. Average vocabulary scores were about the same across the child care and 
school-based cohorts (Figures A42-A43). 

Teacher ratings 

The same types of analysis involving the same groups were conducted for teacher ratings 
of social skills, problem behaviors, and academic competence when PEK child care 
Cohorts 1 and 2 reached kindergarten. Again, we adjusted for differences in student 
characteristics across the groups. 

Comparisons to classmate comparison group 

In the area of academic competence, children in PEK child care Cohorts 1 and 2 were rated 
significantly higher on average than their respective classmate comparison groups without 
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prior preschool or child care center experience, but not significantly differently from 
classmates with such experience. No significant differences were found between PEK 
child care Cohorts 1 and 2 and their respective classmate groups in social skills. In the area 
of problem behaviors, PEK child care Cohort 1 children were rated higher (meaning more 
problem behaviors) on average than both classmate groups, and the difference was 
significant with the group with prior preschool or child care center experience. More 
detailed analyses involving problem behavior subscales indicated that PEK child care 
Cohort 1 children tended to exhibit more externalizing problem behaviors and hyperactivity 
than classmates with other preschool or child care center experiences. On the other hand, 
there were no significant differences in problem behaviors between PEK child care Cohort 
2 and either of the classmate comparison groups (Figures A44-A47). 

Comparisons to school-based PEK 

As on the academic assessments, children who participated in PEK at school sites 
appeared to have some advantages over children who participated in PEK at child care 
sites in their social skills and problem behaviors. In fall of kindergarten, teachers rated 
school-based Cohort 2 students significantly higher than child care Cohort 1 in social 
skills. Specifically, analysis of the social skills subscales found that school-based Cohort 
2 had higher ratings for cooperation and self-control. However, the social skills 
advantage for school-based PEK was not observed the following year in the comparison 
of child care Cohort 2 and school-based Cohort 3. On the other hand, school-based PEK 
had an advantage over both of the child care cohorts in terms of behavior. In fall of 
kindergarten, teachers rated the school-based PEK cohorts significantly lower on average 
in problem behaviors compared to their respective child care cohorts. More specifically, 
analysis of the problem behavior subscales indicated that the school-based cohorts had 
significantly fewer externalizing and hyperactivity problems than the child care cohorts. 

No significant differences were found between school-based PEK and child care PEK in 
the area of academic competence (Figures A48-A5 1). 

Implementation efforts 

This section explores the extent to which PEK’s child care component has been 
implemented as intended. Implementation results are provided through the end of the 
program’s third year of operation in child care settings, 2008-09. Findings presented here 
are organized into the following topics: 

■ Alignment with the Project for Academic Excellence 

■ Language and literacy supports 
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■ Teacher and director satisfaction 

■ Professional development 

■ Implementation of teaching strategies 

■ Teachers’ parent education efforts 

■ Parent involvement 

Alignment with the Project for Academic Excellence 

In summer 2009, an outside observer completed the third-annual classroom observations 
assessing PEK child care centers’ and homes’ alignment with the Project for Academic 
Excellence. A detailed report on these results was prepared by the University of 
Minnesota’s Center for Early Education and Development (CEED) (Hawley, 2009). A 
few key findings are summarized here. 

Overall, PEK child care sites showed instruction and practices that are aligned with the 
Project for Academic Excellence, according to the CEED report. Based on 2009 
observations, most environmental components and routines were implemented to some 
extent across sites. In general, the observations found the child care environments to be 
“literacy rich.” There seemed to be room for improvement, however, in the extent to 
which teachers actively used environmental components throughout the day to promote 
literacy. In the area of routines, the report found that “teachers were fairly consistent 
about putting a particular routine into place but were less consistent in implementing all 
of the components” (Hawley, 2009). 

PEK child care sites attained strong fidelity with the following indicators related to 
classroom routines: 1) morning meeting; 2) read aloud; 3) “ease into the day” routines; 

4) use of shared reading; and 5) routines associated with the “regroup to revisit” portion of 
the day. A few areas of alignment were identified as having “varied fidelity,” meaning 
fidelity was “high in some programs, low in others.” Observations found wide variation in 
practices associated with active learning time, as well as in the extent to which teachers 
differentiated small groups and the number of children included in small groups. Sites also 
varied in their use of interactive writing; their use of transitions, their interactions with 
parents, and their intentional use of conversation to promote vocabulary (Hawley, 2009). 

The report describes how the program’s training and expectations can affect the 
classroom outcomes: 
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For this cohort, coaches seem to have firm ideas about which interactions to 
teach, model, and support with resources. The more explicit the expectations for 
how a routine should be conducted, the higher the implementation fidelity. Areas 
that had less training (interactive writing, for example) exhibited more variability 
in how the routine was implemented (Hawley, 2009). 



The CEED report also describes child care providers’ perceptions of the importance 
of PEK coaching: “Nearly every participant, when asked ‘what makes the biggest 
difference?’ responded with a comment about their relationship with the PEK coach” 
(Hawley, 2009). Further, the report noted that coaching was intentionally tied to other 
aspects of professional development: 

PEK’s combination of training, coaching, written resource, accountability builds 
consistency and clear expectations. In 2009, PEK continued to refine its multi- 
tiered approach that included clearer goals and congruence throughout 
professional development strategies and resources. For instance, coaching visits 
reinforced content that was recently covered in the training sessions. Written 
resources build a shared reference of definitions, descriptions, and expectations 
throughout the learning community (Hawley, 2009). 



Language and literacy supports 

Structured observations also assessed language and literacy supports in PEK child care 
settings. The Early Language Literacy and Classroom Observation tool (ELLCO) is used 
to assess center classrooms, and the similar Child/Home Early Language and Literacy 
Observation (CHELLO) tool is used in family child care homes. A summary of fall 2008 
and spring 2009 observations was prepared by CEED (Hawley, 2009). Overall, PEK 
sites were found to be strong in their support for language and literacy. The spring 2009 
classroom observations indicate that growth occurred across all areas that the ELLCO 
measures, from the availability of books and writing materials to intentional 
implementation of routines and curriculum that support early literacy. Growth in all 
areas is also evident in the CHELLO measures for family child care homes, especially in 
the availability of writing materials throughout the classroom environments, activities 
that promote writing skills, and usefulness of data to monitor children’s progress (Figures 
A52-A53). 

While sites were strong in their supports for language and literacy overall, there were also 
variations among sites. A few areas for improvement were recommended. The observer 
noted that measures of teacher-child interactions at centers and homes that are more 
general in nature, such as classroom climate and behavior management, did not improve 
as much as the other areas. The observer also noted that a lower increase in oral language 
facilitation at child care centers was somewhat unexpected, particularly since the 
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professional development places strong emphasis in this area and that the previous cohort 
of participants showed the highest increase in this area. In fact, the observer saw family 
child care homes teachers more often use advanced vocabulary, explain words to children 
so that they understood, and ask questions that led children to use higher order thinking 
and language skills than center teachers. The observer also felt that center teachers 
needed to make more intentional teaching that connects the literacy-rich environment and 
children’s activities by inviting children to more actively participate in writing centers 
and book reading, for example. Finally, she noted that more family child care providers 
than center teachers used IGDIs to monitor children’s progress and communicate to 
parents (Hawley, 2009). 

Summarizing the ELLCO, CHELLO, and Project for Academic Excellence observations, 
the CEED report concludes as follows: 

In 2009, pre-post ELLCO and CHELLO measures, implementation observations, 
and teacher reports indicate many positive changes have occurred over the past 
year. Child care centers have participated in professional development, made 
changes in their environments, and implemented new teaching strategies and 
interactions. Center directors are key partners in teacher support and PEK 
implementation. Family child care providers have gained knowledge and added 
many early literacy practices, including environmental print, writing experiences, 
and components of Early Childhood Workshop. Resources for Child Caring staff 
used lessons learned from PEK to build school readiness capacity in child care 
settings outside of St. Paul through training and coaching with the support of the 
PEK master coach. While the level of implementation varies, data indicated 
growth and change, particularly in areas where professional development goals 
and resources were most explicit. PEK offers support for teachers, programs, 
communities, and systems to build consistent early learning experiences for 
children across St. Paul and beyond (Hawley, 2009). 



Teacher and director satisfaction 

Overall satisfaction with PEK 

In July 2009 Wilder Research facilitated focus groups for child care teachers and child 
care center directors participating in PEK’s child care component. Feedback gathered 
through the focus groups was intended to inform the program’s work with the second 
year of Cohort 2 child care providers beginning in fall 2009. At the end of their focus 
group session, participants were asked to complete a self-administered questionnaire. 
Participants included 19 child care center teachers and assistant teachers, 13 family child 
care home providers and assistant providers, and 9 child care center directors and 
assistant directors. These respondents included 6 1 percent of center teachers and 
assistant teachers, 72 percent of family child care home providers and assistant providers, 
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and 90 percent of center directors and assistant directors with the program at that time. A 
limitation of the focus groups is that a number of the participating teachers and directors 
had been with PEK for a relatively short period of time and therefore had not been 
exposed to the program during its full implementation in child care settings. 

Key findings and recommendations emerging from the discussions and survey results are 
described in detail in a separate report available from Wilder Research (Broton & Gozali- 
Lee, 2009). Providers were generally positive about their experience with the program 
and had favorable perceptions of the program’s effectiveness. Teachers and center 
directors perceived strong academic gains in children participating in PEK, and felt that 
children were behaving better and more engaged in learning. Teachers said they were 
better able to prepare children for school as a result of participating in PEK. Some 
participants indicated they wanted to continue PEK practices and stay connected to the 
program even after their formal contract with PEK ended. Following are examples of 
participants’ feedback: 

PEK has made such a difference in the children, teachers, and center. Success has 
been so phenomenal. 

-child care center director 

It makes you continue to think about your teaching. They [PEK staff] keep 
giving you ideas - it’s good, it’s great. 

-child care center teacher 

The children are excited to come to daycare and they want to be here because 
they are learning something every day, and the program is so much more 
structured and organized. Children are eager to leam. 

-family child care home provider 

Em happy, kids are happy, parents are happy. It’s a very good program. 

-child care center teacher 

The program has taught me how to build my daycare up. 1 can offer a quality 
program and be competitive with other daycares. 

-family child care home provider 



Suggestions for additional support 

Teachers and directors participating in the focus groups also discussed areas where PEK 
might be able to provide additional support. Key suggestions are summarized here. 
Additional suggestions that pertain to the program’s professional development are 
summarized in the following section on professional development. 

Center teachers indicated they would like more resources and books, and additional 
themes and activities. Center teachers are given the complete Doors to Discovery 
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curriculum to implement, in addition to the PEK child care implementation manual which 
covers the Early Childhood Workshop classroom framework. These teachers seemed 
very favorable about the curriculum, but also indicated it would be nice to have 
additional themes to choose from and, because children can go through them very 
quickly, additional activities. In addition to literacy, some teachers would like to have 
science, arts, and music activities, for example. Lessons and activities specifically for 
special needs children were also requested. Center teachers and directors mentioned the 
need for additional preparation time for lessons, out-of-pocket expenses, and extra 
paperwork as challenges to implementing PEK. 

Some family child care home providers indicated they would like additional opportunities 
to gain ideas from other teachers, and some would like additional adaptations for working 
in a home environment. Program staff have provided monthly training specifically for 
family child care providers and worked extensively with providers to accommodate needs 
associated with working in a home environment. Still, some providers continue to 
struggle with implementing certain aspects of the program. 

Professional development 

Overall satisfaction with professional development 

Overall, teachers and center directors participating in the summer focus groups found 
PEK’s professional development to be very helpful. In the self-administered 
questionnaires completed at the end of the focus groups, eight center directors in 
attendance strongly agreed and one somewhat agreed that participation in PEK 
professional development had a large impact on practices at their child care center. 
Similarly, most child care center teacher strongly agreed that participation in PEK 
professional development had a large impact on their teaching practices, with the rest 
somewhat agreeing. All responding family child care home providers indicated 
agreement with the statement, with all strongly agreeing (Figure A54). 

Teachers also provided positive feedback about PEK’s professional development in the 
focus group discussions. Most teachers commented favorably on both the training and 
coaching. For example, two teachers commented as follow: 

The trainings were very infomiative and so good that I did not want to leave. 

-family child care providers 

My coach helped me with lesson plans. She had me cut down the detail on my 
lessons. It makes sense. 

-child care center teacher 



Project Early Kindergarten evaluation: 
Results through 2008-09 



69 



Wilder Research, September 2009 





During 2008-09, child care directors were asked to be the instructional leaders for the 
implementation of PEK at their centers. Directors attended six months of training prior 
to their teachers attending PEK training to equip them with a solid overview of the 
program. In the focus group, directors expressed that through PEK participation, they 
have changed the way they interact with their teachers. Instead of a sole supervisory role, 
directors indicated that they are now asking purposeful questions about the classroom 
environment and specific activities. As instructional leaders, the directors stated that they 
are more active in the classroom. Additionally, some directors indicated that they are 
more aware of what their teachers are going through and are better able to explain the 
PEK program to parents. Examples of directors’ comments regarding their instructional 
role are included below. 

I think I have better questions now for the teachers. I can ask why they put 
things where - I can ask the puipose of things. 

-child care center director 

They’re teaching with a purpose, and we’re directing with a purpose. We have a 
more active role in the classroom, not just a supervisor role. 

-child care center director 

Implementation of teaching strategies 

Another theme that emerged from the spring focus groups was that center teachers and 
family child care home providers viewed PEK as having positively impacted their 
teaching and their ability to prepare children for kindergarten. In the self-administered 
questionnaires completed at the end of the focus group sessions, all nine directors 
strongly agreed that their center better prepares children for school because of their 
participation in PEK. Almost all of the family child care providers strongly agreed that 
they better prepare children for school because of PEK, while nearly two-thirds of center 
teachers strongly agreed and about one-third somewhat agreed with the statement. Just 
one participant somewhat disagreed with the statement (Figure A55). 

In the focus groups, teachers indicated that the program has improved their focus, 
purpose, and organizational skills, and given them useful strategies to incorporate into the 
classroom. Teachers stated that they like how PEK makes them continue to think about 
their teaching and consider additional ways to become a more effective teacher. 

Directors echoed these comments and stated that their teachers feel more empowered 
from the new infonnation and skills they learned from PEK. Examples of teachers’ 
comments are included below. 
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Before participating in PEK, I was trying to teach and do everything in one day. 
Now, I have learned to break things down and teach a little at a time so that 
children learn and understand. It has made me more organized and structured 
and less stressed out. 

-family child care home provider 

PEK has made me a stronger teacher. 

-child care center teacher 

The curriculum has taught us to be more organized, more structured, and has 
given us the resources and tools needed to teach the children. 

-family child care providers 



Teachers’ parent education efforts 

In March and April 2009 Wilder Research conducted telephone interviews with parents 
of 3- and 4-year-old children participating in PEK at child care centers and family child 
care homes. To be interviewed, parents’ children had to have been enrolled for at least 
two months from September 2008 to March 2009. Parents of 134 children participated in 
the interviews, including 33 children who attended family child care homes and 101 
children who attended child care centers. They represented 73 percent of all children 
attending during 2008-09. 

Almost all interviewed parents said they had heard or were familiar with the fact that 
their provider was working with PEK (100% of parents with children at homes and 95% 
of parents with children at centers). Parents were also asked whether they had received 
and used a variety of infonnation. Asked about the “Talk, Read, Write” and “Help Your 
Child Learn to Read” monthly handouts, most parents of children at family child care 
homes said they received the infonnation (91-94% for each handout). Somewhat smaller 
proportions of center parents said they received these handouts (76-80%). These 
responses were slightly higher than the previous year when 80-83 percent of parents with 
children at homes and 61-69 percent at centers reported they received the handouts. In 
addition to literacy skills, the spring 2009 survey asked parents whether they had received 
“Help Your Child Learn Math” handouts. Compared to reading and writing, fewer 
parents at both homes and centers reported they received the math handouts (57% at 
homes and 64% at centers). 

For both centers and homes, most parents said they received health infonnation (72-86%) 
and information on how to register for kindergarten (80-81%). Somewhat smaller 
proportions of parents reported receiving information on community resources (63-73%). 
In each of these cases, parents who received the infonnation typically said they used it. 
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Asked about what community services they would like to know more about, almost all 
parents (85% of all parents) said they would like infonnation on free or inexpensive 
places for families with young children to go. Between 28 and 43 percent of all parents 
said they would like more infonnation on health care for children and families, Coats for 
Kids, Operation Joy, free tax services, family counseling, and job training for adults. 

Parent involvement 

The spring 2009 parent interview also included questions about parents’ involvement 
in their children’s learning. In general, parents indicated they were involved in their 
children’s learning in a variety of ways, and responses were fairly similar across centers 
and homes, and with the previous year’s parent responses. Almost all parents (92%) 
reported that they read to or look at books with their child everyday or most days, and the 
remaining said they do so once a week (5%) and once in a while (3%). Most parents 
reported that they teach their child new words everyday or most days (84%), 1 1 percent 
said they do so once a week, and 4 percent once in a while. Asked about their support for 
their child’s writing, most parents said they help their child write letters or words 
everyday or most days (74%) or once a week (19%), and almost all said they provide 
their child with writing materials everyday or most days (88%). Asked how frequently 
they take their child to the library, half of parents (50%) indicated once in a while, 34 
percent indicated daily or weekly, and the remaining 16 percent said they never do. 

Parents were also asked about their communication about what their child is learning in 
child care. Almost all parents said they ask their child about what their child is learning 
at child care everyday or most days (97% of home and center parents). However, smaller 
proportions said they talk to their child care provider about what their child is learning 
everyday or most days (67% and 60%, respectively) (Figure 15). 
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15. Parents’ communication about what their children are learning 
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Parents were also asked whether they would be interested in attending workshops on a 
variety of topics related to supporting their children’s learning. The most popular was 
Family Fun Nights, with 72 percent of parents saying they were interested. About half 
of parents said they were interested in attending workshops on helping children leam to 
read (56%), preparing their child for kindergarten (53%), and helping children leam to 
write (53%), as well as workshops providing information about Saint Paul Public Schools 
(48%). For most parents (85%), evening was the best time for parent workshops. 

Parents with children entering kindergarten in the fall were asked questions about their 
child’s preparedness. Almost all (93%) said the child care center or family child care 
home helped prepare their child for kindergarten. Most said their child was registered for 
kindergarten (89%), and that their child had Early Childhood Screening or had a 
screening scheduled (95%). Rates were fairly similar across centers and homes. 
Additionally, most parents whose children were registered reported that their child would 
attend a Saint Paul public school (72%), and that they and their child had visited the 
school their child would attend (89%). 
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Issues for consideration 

The PEK child care component is at an earlier stage of development than the school 
component, having started a year after initial implementation at school sites. Based on 
the findings presented in this report, following are several issues that can be taken into 
consideration in the future planning of PEK’ s child care component. As in the school 
section, some of the issues pertain to PEK staff and some pertain to the researchers 
studying PEK. 

■ Enhancing the community component study. At this point, Wilder Research results are 
available for only the first two cohorts of children participating in the child care 
component of PEK. These results seem promising in some areas and also suggest that 
there is room for improvement. However, due to the small number of children in 
Cohorts 1 and 2, their results are more suggestive than conclusive. We anticipate more 
reliable results in 2009-10 when Cohort 3 child care children begin kindergarten. 
Cohort 3 is larger than the previous cohorts and the children are assessed in both fall of 
their PEK year and fall of kindergarten year. The larger sample size and multiple 
assessments for Cohort 3 should provide more definitive results. Additionally, PEK 
staff are using what they have learned from working with the first cohort of providers 
and made program changes for the second cohort of providers. It will be important for 
the study to examine whether results for children improve over time. 

■ Social skills and problem behaviors. Particular attention may need to be paid to 
improving the socials skills and reducing the problem behaviors of children 
participating in PEK child care programs. When they reached kindergarten, PEK 
child care Cohort 1 children tended to exhibit more externalizing problem behaviors 
and hyperactivity than children who attended PEK at school sites and classmates with 
other preschool or child care center experiences. Cohort 1 children who attended 
PEK schools also tended to exhibit more cooperation and self-control. Similarly, 
Cohort 2 child care children exhibited more problem behaviors than children who 
attended PEK at school sites. Although these differences may in part reflect 
differences in child and family characteristics among the groups, PEK staff can 
consider whether child care teachers could benefit from any additional training on 
how to foster social skills in children. When data on additional children become 
available in the future, it may also be instructive for researchers to explore whether 
there are any differences in social skills between children who participated in PEK at 
home vs. center child care environments, given differences between the two 
environments in the number of children who are together, the number of same-age 
peers, and other characteristics. 
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■ Additional training , resources, and supports. Feedback from the second cohort of 
teachers in summer 2009 focus groups indicated that teachers need more guidance on 
how to connect the PEK instructional concepts to their practices in the classrooms. It 
should be noted that 2008-09 was the first year the second cohort of providers 
participating in PEK. It may be important for PEK coaches to review with teachers 
the implementation manual from time to time so that teachers are comfortable with 
using the manual to guide them through the lessons and activities. It also seems 
important for PEK to provide teachers with a solid overview of the program, 
describing the goals, concepts, and proven benefits of the program to get greater buy- 
in from teachers. 

Additional resources, such as books and materials, were requested. Teachers also 
want additional themes and activities. Opportunities to connect with other providers 
and leam from each other were suggested by family child care home providers. 
Directors may need help in creating a better system to handle extra paper work, such 
as collecting attendance data. 

■ Parent outreach. Parents of children at PEK child care sites reported receiving a 
variety of infonnation from the program, and reported being involved in their child’s 
learning in a number of ways. The program has worked over time to refine and 
expand its parent outreach efforts. The following suggestions based on feedback 
from the parent phone interviews may be helpful in this ongoing process: 

> Parent-teacher communication. Encourage home and center providers to initiate 
conversations with parents about what their child is learning in PEK. 

> Monthly handouts. Work with centers to ensure parents receive the monthly 
handouts. Consider asking center teachers to verbally alert parents to the handouts. 

> Family activities. Provide parents with additional infonnation on free or 
inexpensive places for families with young children. One idea is to provide 
infonnation on library programs, given that a majority of parents said they take 
their child to the library only once in a while or less. PEK may also want to 
consider adding “Family Fun Nights,” since most parents said they would be 
interested in attending those. 

> Parent workshops. Consider offering evening parent workshops that provide 
infonnation on preparing children for kindergarten, helping children learn to 
write, helping children leam to read, and the Saint Paul Public Schools. 

> Kindergarten registration. About 80 percent of parents said they had received 
infonnation on how to register for kindergarten. By the time of the spring phone 
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interviews, most but not all parents with children entering kindergarten in the fall 
had registered their child for kindergarten and had taken their child for Early 
Childhood Screening. In order to reach all parents, PEK may want to provide 
infonnation on kindergarten registration at multiple times during the year, or offer 
a workshop on the topic as suggested above. 

■ Supporting language and literacy. Creating environments that are strong in their 
intentional promotion of literacy is a process, and basic expectations will need to be 
emphasized before higher-level supports can be addressed. From the observations 
conducted by CEED in 2008-09, we know that at the basic level, teachers may need 
help with improving classroom climate and managing behavior. Once these basic 
components are in place, teachers may need help connecting active learning centers to 
lesson themes and encouraging children to make use of book and writing areas. 

Due to variations among sites in their supports for language and literacy, it will be 
important to use site-level data to target support to the needs of individual sites. 
During 2008-09 observations, the observer noted that family child care homes 
providers more often used advanced vocabulary, explained words to children so that 
they understood, and asked questions that led children to use higher order thinking 
and language skills than center teachers did. The observer also felt that center 
teachers need to do more intentional teaching that connects the literacy-rich 
environment and children’s activities by inviting children to more actively participate 
in writing centers and book reading, for example. Finally, she noted that more family 
child care providers than center teachers used IGDIs to monitor children’s progress 
and communicate to parents (Hawley, 2009). 

■ Understanding PEK’ s impact at home vs. center sites. Child care centers and family 
child care homes differ from each other in important ways. This study has the 
potential to contribute useful infonnation about whether differences in these 
environments seem to affect children’s outcomes. When the first two cohorts of PEK 
child care children reached kindergarten, there were not sufficient data available for 
researchers to analyze the results of the two groups separately. However, in the 
future we will be able to combine data from multiple cohorts of child care children to 
examine the results of children attending home vs. center child care settings. 

■ Understanding PEK ’s impact on parents ’ choices. As researchers leam more about 
the results of various settings offering PEK, it may also be instructive to explore how 
the availability of PEK affects parents’ choices. For example, how many PEK 
children at school, child care center, and family child care home sites would not 
otherwise have participated in a preschool program? Do some families with children 
at home sites choose not to enroll their child in another preschool program because 



Project Early Kindergarten evaluation: 
Results through 2008-09 



76 



Wilder Research. September 2009 





their provider participates in PEK? Answers to these questions can be explored 
through the addition of questions to the spring parent surveys and interviews, and may 
yield instructive insights as additional data on the various settings become available. 

■ Positive program changes. PEK should be commended for its efforts to collaborate 
with the community child care partners by offering professional development that is 
rigorous and adaptable to homes and centers environments. Program staff listen to 
the feedback from child care providers. Because homes and centers have different 
needs, the program began offering separate training sessions for each setting in 2008- 
09. To maintain stability in teaching practices and address teacher turnover issue, 
training workshops are extended to assistant teachers and directors. Program staff are 
probably in the best position to determine whether more can be done to accommodate 
needs while maintaining program integrity. 
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Lessons learned 



The Minnesota Early Learning Foundation (MELF) is in the process of gathering 
information on the effectiveness of various early childhood education strategies. 
Ultimately, the foundation intends to make policy recommendations in 201 1. The 
following excerpt from MELF’s 2008 annual report articulates these plans: 

MELF’s mission is to recommend cost-effective strategies for preparing children to 
succeed in school. We are aggressively pursuing this mission by compiling a body 
of knowledge about what works best and most cost-effectively in promoting 
learning readiness among children of low-income families, and families facing 
other challenges. We are weighing the effectiveness of various program models, 
supporting the empowerment of parents, and detemiining the valid short and 
longer-term outcomes and indicators. MELF is taking a ‘systems look’ at the 
early childhood learning and education field in Minnesota (MELF, 2008, p. 1). 

Results from the PEK evaluation will provide valuable information for determining the 
best and most cost-effective strategies for preparing children for school. In addition to 
providing information on the effectiveness of the overall PEK model, the evaluation 
offers insights into what components of the model seem integral and what components 
may need to be strengthened or may be more discretionary. Ultimately, the PEK 
evaluation will also incorporate an analysis that provides information on the cost- 
effectiveness of the overall program. 

This section provides a preliminary list of “lessons learned” in the PEK evaluation that 
may hold policy implications. These include initial lessons about what seems important 
to the program’s success, and what has not worked as well or may be more discretionary. 
Five years after receiving initial program funding and four years after serving the first 
group of children, a number of programmatic successes and challenges have been 
identified. Evaluators will continue modifying and adding to this list as part of the 
program’s ongoing evaluation. 

■ School component’s effectiveness at promoting kindergarten-readiness. At this point, 
there is fairly strong evidence of the effectiveness of the school component in 
preparing children for kindergarten. All three cohorts of children in the school 
component showed significant academic and social advantages over their peers when 
they reached kindergarten, and their results are stronger with each successive year. 
Less is known at this point about the effectiveness of the child care component, or 
about home vs. center environments within that component. 

■ Importance of professional development component. Similar to the Project for 
Academic Excellence, PEK emphasizes intensive, ongoing professional development. 
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To date, teacher reports validate the importance of the professional development 
component. Teachers have credited the program’s professional development with 
impacting their teaching practices. Within this component, coaching seems to be an 
important means for ensuring teachers understand and can implement what is learned in 
training, and for providing accountability for expectations communicated in trainings. 

■ Importance of emphasis on early literacy skills. Based on results available to date, 
PEK’s strong emphasis on early literacy skills seems to be a key program component. 
When they reached kindergarten, PEK school-based children showed advantages in 
vocabulary and early reading and writing compared to similar children who had chosen 
but not yet received PEK. Structured classroom assessments found that overall, PEK 
school sites meet standards for promoting language and literacy in the classroom. 

■ Importance of administrative buy-in. The program’s integration into schools and 
expansion across the district have required the support and buy-in of school principals 
and district administrators. As the “instructional leader” of PEK at their school, 
principals are involved in classrooms and oversee classrooms’ implementation of the 
program model. The program has recognized a need for similar buy-in at child care 
centers, and assigned the second cohort of center directors with a comparable role. At the 
district level, leadership within the Office of Academics has been actively involved in the 
consolidation of 4-year-old programs under the PEK model. In the larger community, 
leadership at Resources for Child Caring has championed the program model with child 
care providers and initiated similar programs with four other school districts. 

■ Inclusion of parent involvement component. In 2009 survey, several principals at the 
PEK schools mentioned that PEK helps educate parents about the importance of 
parent involvement in their children’s education. Principals also appreciated building 
the connections with parents early on. At this point, it is difficult to know the relative 
importance of the parent involvement component to the results we have seen in 
children. Results indicate that parents are involved in their children’s learning in a 
number of ways and that there also may be room for improvement in some areas. 
Although it may be difficult to make claims about the parent involvement component 
based on data currently available from this study, other research validates the 
inclusion of this component. Research indicates that strong center-based early 
childhood programs involving parents can impact parenting in ways that affect school 
readiness (Brooks-Gunn & Markman, 2005). 

■ Importance of linkages with early elementary instruction. Early results from the 
school component suggest that program strategies need to address the program’s 
implications for early elementary grades. Between kindergarten and first grade, 
differences between former PEK students and their classmates narrowed on average. 
Principals described a need for differentiated instruction in kindergarten to meet the 
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varying needs of incoming children, including relatively high skill levels of children 
who attended PEK. Toward this end, PEK leaders have begun working with schools 
to equip kindergarten teachers to differentiate their instruction based on children’s 
incoming skill levels. 

■ Questions about program dosage. Children participating in PEK’s school-based 
program attend half days five days a week. In contrast, PEK’s child care component 
and the PEK-Early Reading First program offer full days of programming. Wilder 
Research’s evaluations of all three may provide a unique opportunity to explore how 
the number of hours a child attends (i.e., program dosage) seems to affect academic 
and social outcomes. In the future, results from PEK’s child care component may 
provide insights in this area. Depending on funding, Wilder Research may also 
consider comparing results of PEK-Early Reading First students with those of PEK 
students in the future. 

■ Gauging program cost-effectiveness. Ultimately, we intend to provide infonnation on 
the cost-effectiveness of the PEK program. The intent is that once sufficient data are 
available, Wilder Research’s economists will conduct an analysis on the cost- 
effectiveness of the overall program. This analysis will compare the relative 
effectiveness of PEK and other similar programs for preschool-age children in relation 
to their costs. 

In addition to preliminary lessons developed by researchers based on evaluation results, 

program staff have also suggested lessons they perceive as important based on their work 

with the program: 

■ Using data to drive instruction. PEK teachers use Work Sampling System 
assessments and some subtests of Phonological Awareness Literacy Screening (in 
schools) and Individual Growth and Development Indicators (in schools and child 
care settings) to monitor children’s progress over the course of their PEK year. 
Program staff perceive this progress monitoring as an important tool for 
differentiating instruction based on individual students’ needs. According to program 
staff, these assessments can also be used to motivate teachers by demonstrating 
students’ progress over time. Evaluation results also suggest that teachers value the 
data received from these assessments. 

■ Establishing high expectations. Program staff also perceive a key component of the 
program to be its establishment of clear and high expectations for teachers and 
students. The program emphasizes academic rigor and the development of critical 
thinking skills. Program staff perceive teachers’ and students’ awareness of specific 
program expectations to be key to the progress they have made. 
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As previously noted, this list represents a preliminary compilation of lessons learned 
from the PEK evaluation that may be useful to practitioners and policymakers making 
decisions about planning and funding early childhood programs. Over the next few 
years, data gathered through the study will be used to modify and expand this list. 
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Community-based PEK 
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School-based PEK 



A1. PEK school component. “Birthday cutoff” method illustration, assuming effective treatment 

Comparison (Younger Children) Treatment (Older Children) 




Regression lines: 

• Estimated relationship between age and test score for each cohort 

• The gap between the lines at the “cut-off birth date” is the estimated treatment effect 
(impact of PEK) 

Note: The PEK school component uses the “birthday cutoff” method. In this method , treatment and comparison groups are defined by whether a 

child's fourth birthday falls before or on/after September 1, the birthday cutoff date used to determine eligibility for PEK. For students attending PEK in 2005- 
06, the treatment group consists of children who enrolled in PEK in fall 2005 and whose fourth birthdays, therefore, fell before September 1, 2005 (Cohort 1). 
The comparison group consists of children who entered PEK a year later in fall 2006 and whose fourth birthdays fell on/after September 1, 2005, but before 
September 1, 2006 (Cohort 2). Upon kindergarten entry, the treatment group (Cohort 1 in this case) is compared to the comparison group which is just 
entering PEK (Cohort 2 in this case). The comparison is carried out using a regression-discontinuity research design in which two regression lines estimating 
test scores by age are developed, one for the treatment group and one for the comparison group. The regression-discontinuity approach assumes that a 
child who just made the age cutoff and a child who just missed it have similar characteristics, except that the former child has received the treatment (PEK) 
while the latter child has not. Given this assumption, the estimated test score difference at the cutoff date should provide an unbiased estimate of the 
treatment effect (Barnett et at., 2005; Gormley et at., 2005). For students attending PEK in 2006-07, the treatment group consists of Cohort 2 and the 
comparison group consists of Cohort 3. 
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A2. PEK school component. Demographic characteristics of Cohorts 1, 2, and 3 in fall of their PEK 



year 

Characteristics 




Cohort 1 
(fall 2005) 
N=325-326 


Cohort 2 
(fall 2006) 
N=324-329 


Cohort 3 
(fall 2007) 
N=312 


Gender 


Female 


51% 


47% 


49% 




Male 


49% 


53% 


51% 


Race/ethnicity 


American Indian 


3% 


4% 


4% 




Asian 


27% 


24% 


30% 




Latino 


20% 


16% 


18% 




Black 


31% 


39% 


33% 




White 


19% 


17% 


15% 


Home language 


English 


50% 


55% 


52% 




Hmong 


24% 


20% 


22% 




Spanish 


17% 


13% 


13% 




Other 


9% 


12% 


12% 


ELL 


Yes 


49% 


45% 


48% 




No 


51% 


55% 


52% 


Free/reduced-price lunch eligibility 


Eligible 


61% 


74% 


71% 




Ineligible 3 


39% 


26% 


29% 


Special Education 


Yes 


12% 


12% 


11% 




No 


88% 


88% 


89% 


In target population 13 


Yes 


79% 


88% 


87% 




No 


21% 


12% 


13% 



a Includes families who were not eligible for free or reduced-price lunch as well as families who did not apply. 

b Child is in one or more of the following categories: eligible for free or reduced-price lunch , ELL, or receives Special Education services. 



Notes: This figure presents demographic data from fall of the PEK year for children who were assessed at that time. The “Ns” in this figure may differ 

somewhat from those in other figures in this report and previous reports. One reason is that for purposes of this demographic profile, we included children 
who were assessed in Spanish and therefore excluded from analyses of results. Another reason is that a few children who were tested were subsequently 
excluded from results because their birth date did not fall within the range for their cohort based on the program’s birthday cutoff date. There could also be 
some slight differences in “Ns ” between this and other figures based on children being assessed with either the Peabody or Woodcock-Johnson, but not both. 
It is important to note that methods for obtaining PEK children's demographic characteristics changed in 2006 after the district introduced a new application 
process for 4-year-old programs. It should also be noted that children ’s demographic characteristics, such as their free- or reduced-price lunch status, can 
change over time. 
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A3. PEK school component. Demographic characteristics of Cohort 1 and comparison group 
(preschool and no preschool) in fall 2006 (kindergarten), using fall 2006 demographic data 









Comparison group 3 


Characteristics 




Cohort 1 
(n=263) 


With preschool/ 
child care 
center 
(n=1 56) 


Without 

preschool/ child 
care center 
(n=100) 


Gender 


Female 


52% 


47% 


54% 




Male 


48% 


53% 


46% 


Race/ethnicity 


American Indian 


3% 


7% 


5% 




Asian 


27% 


19% 


37% 




Latino 


18% 


16% 


17% 




Black 


31% 


40% 


19% 




White 


21% 


19% 


22% 


ELL 


Yes 


47% 


40% 


50% 




No 


53% 


60% 


50% 


Free/reduced-price lunch eligibility 


Eligible 


65% 


89 %* 


87%* 




Ineligible 


35% 


11% 


13% 


Special Education 


Yes 


14% 


15% 


3% 




No 


86% 


85% 


97% 



a The comparison group was divided into two groups - those who attended preschool , Head Start , or a child care center prior to attending kindergarten , 

and those who did not. 

* p< 05, compared to Cohort 1. 
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A4. PEK school component. Demographic characteristics of Cohort 2 and comparison group 
(preschool and no preschool) in fall 2007 (kindergarten), using fall 2007 demographic data 









Comparison group 3 


Characteristics 




Cohort 2 
(n=266) 


With preschool/ 
child care 
center 
(n=139) 


Without 

preschool/ child 
care center 
(n=145) 


Gender 


Female 


47% 


42% 


48% 




Male 


53% 


58% 


52% 


Race/ethnicity 


American Indian 


5% 


1% 


2% 




Asian 


24% 


22% 


38%* 




Latino 


17% 


25% 


13% 




Black 


36% 


40% 


32% 




White 


18% 


12% 


15% 


ELL 


Yes 


44% 


53% 


46% 




No 


56% 


47% 


54% 


Free/reduced-price lunch eligibility 


Eligible 


56% 


68% 


55% 




Ineligible 


44% 


32% 


45% 


Special Education 


Yes 


17% 


12% 


* 

sO 

CO 




No 


83% 


88% 


97% 



a The comparison group was divided into two groups - those who attended preschool , Head Start , or a child care center prior to attending kindergarten , 
and those who did not. Children with missing data on preschooi/chiid care experience were included in the no preschool/chiid care center group. 

* p< 05, compared to Cohort 2. 
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A5. PEK school component. Demographic characteristics of Cohort 3 and comparison group 
(preschool and no preschool) in fall 2008 (kindergarten), using fall 2008 demographic data 









Comparison group 3 


Characteristics 




Cohort 3 
(n=235) 


With preschool/ 
child care 
center 
(n=1 56) 


Without 

preschool/ child 
care center 
(n=79) 


Gender 


Female 


48% 


49% 


46% 




Male 


52% 


51% 


54% 


Race/ethnicity 


American Indian 


5% 


6% 


8% 




Asian 


28% 


14%* 


39% 




Latino 


20% 


22% 


24% 




Black 


34% 


44% 


18%* 




White 


14% 


14% 


11% 


ELL 


Yes 


50% 


44% 


62% 




No 


50% 


56% 


38% 


Free/reduced-price lunch eligibility 


Eligible 


57% 


77%* 


76%* 




Ineligible 


43% 


23% 


24% 


Special Education 


Yes 


10% 


14% 


3% 




No 


90% 


86% 


97% 



a The comparison group was divided into two groups - those who attended preschool , Head Start , or a child care center prior to attending kindergarten , 
and those who did not. 

* p< 05, compared to Cohort 3. 
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A6. PEK school component. Academic test standard score change for PEK students from fall of 
PEK to fall of kindergarten 







Mean standard scores 3 




Test 


Number 

assessed 


PEK 

(fall 2005) 


Kindergarten 
(fall 2006) 


Change 15 


Cohort 1 




Peabody Picture Vocabulary Test III 


253 


88.1 


91.9 


+3.8*** 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


250 


97.5 


102.2 


+4.7*** 


Spelling (writing) 


251 


99.6 


102.8 


+3.2*** 


Applied Problems (math) 


245 


95.1 


94.4 


-0.7 


Cohort 2 


PEK 

(fall 2006) 


Kindergarten 
(fall 2007) 


Change 15 


Peabody Picture Vocabulary Test III 


266 


86.2 


92.1 


+5.9*** 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


263 


97.2 


103.2 


+6.0*** 


Spelling (writing) 


265 


94.7 


104.1 


+9.4*** 


Applied Problems (math) 


251 


92.0 


95.0 


+3.0*** 


Cohort 3 


PEK 

(fall 2007) 


Kindergarten 
(fall 2008) 


Change 15 


Peabody Picture Vocabulary Test III 


220 


88.8 


96.0 


+7.2*** 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


217 


98.1 


107.6 


+9.4*** 


Spelling (writing) 


219 


98.1 


110.6 


+12.6*** 


Applied Problems (math) 


211 


96.4 


98.0 


+1.7 



Note: The average number of days between the fall of preschool and fall of kindergarten testing periods has varied somewhat by cohort , ranging from 

375 days for Cohort 2 to 390 days for Cohort 1 and 435 days for Cohort 3. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. These scores are age-standardized , meaning 
that no change in scores from one year to the next indicates normative progress , positive change indicates accelerated progress , and negative change 
indicates slower progress in comparison to one’s peers. 

b Fall of kindergarten score minus fall of PEK score. 

*** p<001 
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A7. PEK school component. Academic test age-equivalency change for PEK students from fall of 
PEK to fall of kindergarten 







Mean age-equivalency scores 
(years-months) 


Test 


Number 

assessed 


PEK 

(fall 2005) 


Kindergarten 
(fall 2006) 


Change 


Cohort 1 




Peabody Picture Vocabulary Test III 


253 


3-09 


5-00 


+15 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


250 


4-08 


5-10 


+14 months 


Spelling (writing) 


251 


4-06 


5-11 


+17 months 


Applied Problems (math) 


245 


4-03 


5-03 


+12 months 


Cohort 2 


PEK 

(fall 2006) 


Kindergarten 
(fall 2007) 


Change 


Peabody Picture Vocabulary Test III 


266 


3-06 


5-00 


+18 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


263 


4-08 


6-00 


+16 months 


Spelling (writing) 


265 


4-06 


5-11 


+17 months 


Applied Problems (math) 


251 


4-03 


5-03 


+12 months 


Cohort 3 


PEK 

(fall 2007) 


Kindergarten 
(fall 2008) 


Change 


Peabody Picture Vocabulary Test III 


220 


3-06 


5-06 


+24 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


217 


4-08 


6-03 


+19 months 


Spelling (writing) 


219 


4-06 


6-04 


+22 months 


Applied Problems (math) 


211 


4-03 


5-07 


+16 months 



Note: The average number of days between the fall of preschool and fall of kindergarten testing periods has varied somewhat by cohort , ranging from 

375 days for Cohort 2 to 390 days for Cohort 1 and 435 days for Cohort 3. 
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A8. Change in Peabody standard scores: Cohort 3 fall 2007 to fall 2008 by student characteristics 

Mean standard scores (standard deviation) 

Change 







N 


Fall 2007 


Fall 2008 


score 3 


Race b 


American Indian 


11 


96.73 (12.69) 


97.00 (10.46) 


+0.27 




Asian 


66 


77.11 (21.96) 


89.58 (13.82) 


+12.47 




Hispanic 


31 


90.61 (12.65) 


95.48 (17.71) 


+4.87 




African American 


79 


90.63 (13.24) 


97.08 (9.73) 


+6.44 




Caucasian 


33 


103.24 (15.02) 


106.36 (11.68) 


+3.12 


ELL status c 


Yes 


103 


80.41 (19.75) 


90.92 (14.88) 


+10.51 




No 


117 


96.13 (13.86) 


100.45 (10.78) 


+4.32 


Eligible for free 


Yes 


122 


86.59 (18.85) 


95.04 (12.07) 


+8.45 


or reduced-price lunch 


No 


98 


91.48 (17.96) 


97.17 (15.46) 


+5.69 


Special education status 


Yes 


24 


89.38 (18.32) 


96.79 (9.72) 


+7.42 




No 


196 


88.69 (18.66) 


95.89 (14.12) 


+7.20 



Note: Includes only students with fall 2007 and fall 2008 assessments. 

a 2008 standard score minus 2007 standard score. 

b Significant differences in change scores between Asians and American Indians (p<.05), Asians and Hispanics (p<.05), Asians and African Americans 
(p<.05), and Asians and Caucasians (p<.01). 

0 Significant difference in change scores between ELL andnon-ELL students (p<.001). 
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A9. Change in Letter-Word Identification standard scores: Cohort 3 fall 2007 to fall 2008 by 
student characteristics 

Mean standard scores (standard deviation) 







N 


Fall 2007 


Fall 2008 


Change 

score 3 


Race 


American Indian 


11 


93.55(11.77) 


99.64 (11.55) 


+6.09 




Asian 


65 


96.89(15.36) 


109.11 (9.75) 


+12.22 




Hispanic 


30 


93.10(11.04) 


106.50 (12.67) 


+13.40 




African American 


78 


100.18 (16.50) 


106.65 (13.05) 


+6.47 




Caucasian 


33 


101.91 (15.15) 


110.39 (11.87) 


+8.48 


ELL status 


Yes 


100 


99.40(16.45) 


110.26 (11.69) 


+10.86 




No 


117 


97.07(14.12) 


105.29 (11.76) 


+8.22 


Eligible for free 


Yes 


121 


98.72(15.31) 


106.70 (13.47) 


+7.98 


or reduced- price lunch 


No 


96 


97.42(15.22) 


108.69 (9.70) 


+11.27 


Special education status b 


Yes 


24 


100.13 (15.02) 


104.63 (12.05) 


+4.50 




No 


193 


97.90(15.30) 


107.95 (11.93) 


+10.05 



Note: Includes only students with fall 2007 and fall 2008 assessments. 

a 2008 standard score minus 2007 standard score. 

b Significant difference in change scores between students with special needs and other students (p< 05). 
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A10. Change in Spelling standard scores: Cohort 3 fall 2007 to fall 2008 by student characteristics 

Mean standard scores (standard deviation) 

Change 

N Fall 2007 Fall 2008 score 3 


Race 


American Indian 


11 


94.91 (11.93) 


104.45 (9.05) 


+9.55 




Asian 


66 


99.62 (13.65) 


111.76 (11.34) 


+12.14 




Hispanic 


30 


99.77 (9.45) 


109.83 (11.78) 


+10.07 




African American 


79 


96.65 (14.49) 


109.87 (11.74) 


+13.23 




Caucasian 


33 


97.79 (14.61) 


112.88 (13.30) 


+15.09 


ELL status 


Yes 


102 


101.36 (13.10) 


112.71 (11.99) 


+11.34 




No 


117 


95.17 (13.27) 


108.79 (11.37) 


+13.62 


Eligible for free 


Yes 


122 


97.55 (14.51) 


110.45 (12.72) 


+12.90 


or reduced- price lunch 


No 


97 


98.69 (12.21) 


110.82 (10.60) 


+12.13 


Special education status 


Yes 


24 


93.96 (15.53) 


105.25 (12.64) 


+11.29 




No 


195 


98.65 (13.21) 


111.28 (11.56) 


+12.72 



Note: Includes only students with fall 2007 and fall 2008 assessments. 

a 2008 standard score minus 2007 standard score. 
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All. Change in Applied Problems standard scores: Cohort 3 fall 2007 to fall 2008 by student 
characteristics 

Mean standard scores (standard deviation) 







N 


Fall 2007 


Fall 2008 


Change 

score 3 


Race b 


American Indian 


11 


94.36(15.83) 


95.09 (13.66) 


+0.73 




Asian 


60 


91.78(18.35) 


97.10 (10.25) 


+5.32 




Hispanic 


30 


95.77(15.18) 


97.97 (10.37) 


+2.20 




African American 


78 


96.79(10.86) 


97.22 (9.20) 


+0.42 




Caucasian 


32 


105.22 (15.06) 


102.81 

(11.56) 


-2.41 


ELL status 


Yes 


96 


94.06(17.48) 


97.31 (10.44) 


+3.25 




No 


115 


98.30 (12.92) 


98.63 (10.39) 


+0.32 


Eligible for free 


Yes 


117 


94.82(15.67) 


96.71 (10.04) 


+1.89 


or reduced- price lunch 


No 


94 


98.31 (14.63) 


99.67 (10.69) 


+1.36 


Special education status 


Yes 


22 


92.41 (19.73) 


93.86 (14.20) 


+1.45 




No 


189 


96.84(14.67) 


98.51 (9.81) 


+1.68 



Note: Includes only students with fall 2007 and fall 2008 assessments. 

a 2008 standard score minus 2007 standard score. 

b Significant differences in change scores between Asians and Caucasians (p<.05). 
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A12. PEK school component. PEK effect sizes 3 using birthday cutoff method compared to other 
studies 

PreK in PreK in PreK in New 

five Tulsa, PreK in Mexico; Hustedt 

PEK states; Oklahoma; Arkansas; et al. 

Barnett Gormley Hustedt 

Assessment et al., et al., et al., 



instrument 


2005-06 b 


2006-07° 


2005 


2005 


2007 


2007 


2008 


Peabody Picture 
Vocabulary Test 


.69 


.58 


.26 




.36 


.36 


.25 


W-J Letter-Word 
Identification (reading) 


.75 


.71 




.79 








W-J Spelling (writing) 


.96 

(,69 d ) 


.77 

(1.02 e ) 




.64 








W-J Applied Problems 
(math) 


.88 

(,67 d ) 


.06 f 

(,35 e ) 


.28 


.38 


.24 


.39 


.50 



Note: Caution is needed in interpreting Cohort 1 and Cohort 2 resuits as they may be misleading due to baseline test score differences in the cohorts 

compared using the birthday cutoff method. These differences at baseline tend to inflate the effect sizes for Cohort 1 and diminish the effect sizes for Cohort 
2 for most of the measures. A crude adjustment has been made to compensate for these differences in some cases (as indicated above) where they were 
statistically significant, l/l/fe are currently developing a more appropriate statistical adjustment. Additionally, it is important to note that PEK effect sizes were 
calculated based on the standard deviation for the pooled treatment and comparison group, whereas effect sizes in the other studies were calculated based 
on the standard deviation for the comparison group only. 

a Small effect = 0.2, medium effect = 0.5, large effect = 0.8. Effect sizes are calculated using Cohen's d (1988). 
b The effect of PEK is based on the comparison between Cohort 1 and Cohort 2 in fall 2006 (see Mueller & Gozali-Lee, 2007). 

c The effect of PEK is based on the comparison between Cohort 2 and Cohort 3 in fall 2001. 

d Effect size adjusted for differences between Cohorts 1 and 2 at baseline (fall of PEK year). 

e Effect size adjusted for differences between Cohorts 2 and 3 at baseline (fall of PEK year). 

f No statistically significant difference at the birthday cutoff. 
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A13. PEK school component (fall 2006). Academic test age-equivalency scores 3 at the birthday 
cutoff point (estimate of the effect of PEK on Cohort 1 students based on birthday cutoff 
method) 



Just missed birthday cutoff Just made cutoff 



Assessment instrument 


(Cohort 2) 


(Cohort 1) 


Difference 


Peabody Picture Vocabulary Test 


3-09 


4-09 


12 months 


W-J Letter-Word Identification (reading) 


4-11 


5-07 


8 months 


W-J Spelling (writing) 


4-06 


5-06 


12 months 
(9 months' 3 ) 


W-J Applied Problems (math) 


4-03 


5-01 


10 months 
(6 months' 3 ) 


Note: The expected age equivalency score is 5 years, 0 months at the birthday cutoff based on national norms. 

a In years and months. 

b Adjusted for differences between Cohorts 1 and 2 at baseline (fall of PEK year). 





A14. PEK school component (fall 2007). Academic test age-equivalency scores 3 at the birthday 
cutoff point (estimate of the effect of PEK on Cohort 2 students based on birthday cutoff 
method) 


Assessment instrument 


Just missed birthday cutoff 
(Cohort 3) 


Just made cutoff 
(Cohort 2) 


Difference 


Peabody Picture Vocabulary Test 


3-11 


4-09 


10 months 


W-J Letter-Word Identification (reading) 


5-01 


5-07 


6 months 


W-J Spelling (writing) 


4-09 


5-06 


9 months 
(12 months' 3 ) 


W-J Applied Problems (math) 


4-08 


4-11 


3 months 0 
(4 months' 3 ) 



Note: The expected age equivalency score is 5 years, 0 months at the birthday cutoff based on national norms. 

a In years and months. 

b Adjusted for differences between Cohorts 2 and 3 at baseline (fall of PEK year). 
c This difference is not statistically significant based on the regression discontinuity (birthday cutoff) analysis. 
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A15. PEK school component. Studies that use the birthday cutoff method (continues on following page) 



A. Program features 


PEK 

2005-06 and 2006-07 


Barnett et al., 
2005 


Gormley et al., 
2005 


Hustedt et al., 
2007 


Hustedt et al., 2007 and 2008 


Location(s) 


Saint Paul, Minnesota 


Michigan, New 
Jersey, Oklahoma, 
South Carolina, 
West Virginia 


Tulsa, Oklahoma 


Arkansas 


New Mexico 


Funding 


school district funding 
plus private grant 


state-funded 


state-funded 


state-funded 


state-funded 


Sites 


public schools 


public schools and 
private centers 


public schools 


public schools and 
private centers 


public schools and private centers 


Provider education 


All are licensed 
teachers with four- 
year college degrees 
plus preschool 
certification 


Nearly all are 
teachers with four- 
year college 
degrees with an 
early childhood 
specialization 


All teachers have 
four-year college 
degrees plus 
certification in 
early childhood 
education 


Nearly all (94%) 
are teachers with 
at least a four-year 
college degree 


Lead teachers at each site must 
have four-year college degrees and 
certification in early childhood 
education within 5 years of becoming 
PreK site. In spring 2006, 71% of 
lead teachers responding to a survey 
reported having a bachelor’s degree 


Length of day 


half-day 


Varies 


Varies 


- 


- 


Teacher a /child ratio 


1:10 


1:8 to 1:10 


1:10 or less 


- 


1:10 


Maximum class size 


20 


15 to 20 


20 


- 


20 


Target low-income or at-risk 


Yes 


Varies 


No 


Yes 


Yes 
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A15. PEK school component. Studies that use the birthday cutoff method (continued) 



B. Characteristics of study 
samples 


PEK Cohorts 
1 &2 

2005-06 


PEK Cohorts 
2 & 3 

2006-07 


Barnett et al. 
2005 


Gormley et al. 
2005 b 


Hustedt et al. 
2007 


2007 


Hustedt et al. 

2008 


Sample size 










Treatment 


263 


268 


2,728 


1,461 


504 


382 


405 


Control 


319 


296 


2,550 


1,567 


407 


504 


519 


Gender 










Female 


49% 


48% 


52% 


48% 


48% 


49% 


54% 


Male 


51% 


52% 


48% 


52% 


52% 


51% 


46% 


Race/ethnicity 










American Indian 


4% 


4% 


3% 


9% 


<1% 


28% 


19% 


Asian 


25% 


28% 


2% 


1% 


1% 


1% 


2% 


Latino 


15% 


15% 


21% 


14% 


6% 


56% 


57% 


Black 


36% 


36% 


25% 


39% 


36% 


1% 


2% 


White 


19% 


16% 


47% 


36% 


57% 


10% 


19% 


Other 


- 


- 


- 


- 


- 


2% 


<1% 


Free/reduced-price lunch 










Eligible 


69% 


63% 


- 


66% 


- 


- 


- 


Ineligible 


31% 


37% 


- 


34% 


- 


- 


- 


Age upon PreK entry 


4 


4 


4 


4 


4 


4 


4 



Note: Demographic characteristics are provided for the combined treatment and control groups. 

a Includes certified teachers and teaching assistants. 

b Demographic breakdowns for the combined treatment and control groups are approximations calculated from data provided in the published study. 
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A16. PEK school component (fall 2006). Academic test standard scores in kindergarten: PEK 
Cohort 1 vs. kindergarten classmates 









Mean standard scores 3 








Kindergarten classmates' 3 


Assessment 




PEK 
Cohort 1 


With 

preschool/ 
child care 
center 


Without 
preschool/ 
child care 
center 


Peabody Picture Vocabulary Test III 


Mean 


91.47 


87.40 


83.60 




Adjusted mean 0 


91.02 


86.95** 


85.43** 




Number assessed 


263 


143 


99 


Woodcock-Johnson Tests of 
Achievement III 










Letter-Word Identification (reading) 


Mean 


101.89 


98.79 


95.23 




Adjusted mean 0 


101.67 


99.84 


94.29*** 




Number assessed 


263 


142 


99 


Spelling (writing) 


Mean 


102.25 


99.75 


97.89 




Adjusted mean 0 


102.05 


101.02 


96.59*** 




Number assessed 


263 


143 


99 


Applied Problems (math) 


Mean 


93.93 


91.45 


88.98 




Adjusted mean 0 


93.70 


91.88 


88.97** 




Number assessed 


262 


142 


98 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 



b Kindergarten classmates were divided into two groups - those who attended preschool , Head Start or a child care center prior to attending kindergarten , 
and those who did not. 

c Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

** p<.01, compared to PEK Cohort 1. 

*** p<001, compared to PEK Cohort 1. 
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A17. PEK school component (fall 2007). Academic test standard scores in kindergarten: PEK 
Cohort 2 vs. kindergarten classmates 



Mean standard scores 3 



Test 


PEK Cohort 2 


Kindergarten classmates' 3 

With preschool/ Without preschool/ 
child care center child care center 


Peabody Picture Vocabulary Test III 




Mean 


92.1 


85.7 


83.1 


Adjusted mean 0 


91.6 


86.6** 


83.1*** 


Number assessed 


266 


139 


145 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 




Mean 


103.1 


98.0 


96.3 


Adjusted mean 0 


103.4 


98.5*** 


95.3*** 


Number assessed 


266 


139 


145 


Spelling (writing) 




Mean 


104.1 


99.9 


97.2 


Adjusted mean 0 


104.7 


100.2** 


95.9*** 


Number assessed 


266 


139 


145 


Applied Problems (math) 




Mean 


94.5 


91.4 


87.9 


Adjusted mean 0 


94.8 


91.8* 


87.1*** 


Number assessed 


266 


139 


140 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 



b Kindergarten classmates were divided into two groups - those who attended preschool , Head Start or a child care center prior to attending kindergarten , 
and those who did not. 

c Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

* p<.05, compared to PEK Cohort 2. 

** p<.01, compared to PEK Cohort 2. 

*** p<001, compared to PEK Cohort 2. 
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A18. PEK school component (fall 2008). Academic test standard scores in kindergarten: PEK 
Cohort 3 vs. kindergarten classmates 



Mean standard scores 3 



Test 


PEK Cohort 3 


Kindergarten classmates' 3 

With preschool/ Without preschool/ 
child care center child care center 


Peabody Picture Vocabulary Test III 




Mean 


94.6 


87.6 


81.1 


Adjusted mean 0 


94.7 


87.0*** 


81.9*** 


Number assessed 


235 


152 


78 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 




Mean 


106.9 


102.3 


98.4 


Adjusted mean 0 


106.8 


103.0*** 


97.3*** 


Number assessed 


234 


152 


78 


Spelling (writing) 




Mean 


110.0 


105.1 


101.1 


Adjusted mean 0 


109.7 


106.0** 


99.9*** 


Number assessed 


234 


152 


78 


Applied Problems (math) 




Mean 


96.7 


90.1 


84.6 


Adjusted mean 0 


96.7 


90.6*** 


83.9*** 


Number assessed 


234 


152 


78 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Kindergarten classmates were divided into two groups - those who attended preschool , Head Start or a child care center prior to attending kindergarten , 

and those who did not. 

c Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

* p<.05, compared to PEK Cohort 3. 

** p<.01, compared to PEK Cohort 3. 

*** p<001, compared to PEK Cohort 3. 
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A19. PEK school component. PEK academic test effect sizes in kindergarten: PEK students vs. 
kindergarten classmates 



Test 


Estimated size of PEK effects 3 


Cohort 1 


Cohort 1 vs. 
preschool 
comparison group 


Cohort 1 vs. no 
preschool 
comparison group 


Peabody Picture Vocabulary Test III 


.23 


.30 


Woodcock-Johnson Tests of Achievement III 


Letter-Word Identification (reading) 


.15 


.61 


Spelling (writing) 


.08 


.44 


Applied Problems (math) 


.15 


.37 


Cohort 2 


Cohort 2 vs. 
preschool 
comparison group 


Cohort 2 vs. 
no preschool 
comparison group 


Peabody Picture Vocabulary Test III 


.30 


.49 


Woodcock-Johnson Tests of Achievement III 


Letter-Word Identification (reading) 


.49 


.77 


Spelling (writing) 


.36 


.73 


Applied Problems (math) 


.27 


.60 


Cohort 3 


Cohort 3 vs. 
preschool 
comparison group 


Cohort 3 vs. 
no preschool 
comparison group 


Peabody Picture Vocabulary Test III 


.49 


.70 


Woodcock-Johnson Tests of Achievement III 


Letter-Word Identification (reading) 


.32 


.80 


Spelling (writing) 


.31 


.90 


Applied Problems (math) 


.52 


.95 



a Effect size was calculated using Cohen ’sd(1 988): the difference between the adjusted means of Cohort 1 and the comparison group divided by the 
pooled standard deviation of the two groups (using standard scores). Small effect = 0.2, medium effect = 0.5, large effect = 0.8. These results are 
based on adjustments for demographic (gender, age, race/ethnicity, free/reduced-price lunch eligibility, ELL status, and Special Education status) and 
test date differences of the groups being compared. 
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A20. PEK school component. Academic test age-equivalency scores 3 in kindergarten: PEK students 
vs. kindergarten classmates 



Mean adjusted b age-equivalency scores 
(years-months) 



Test 


Kindergarten classmates c 


Cohort 1 


PEK Cohort 1 
(N=262-3) 


With preschool/ 
child care center 
(N=142-3) 


Without preschool/ 
child care center 
(N=98-9) 


Peabody Picture Vocabulary Test III 


4-11 


4-08 


4-06 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


5-10 


5-09 


5-06 


Spelling (writing) 


5-09 


5-09 


5-06 


Applied Problems (math) 


5-03 


5-01 


4-11 


Cohort 2 


PEK Cohort 2 
(N=266) 


With preschool/ 
child care center 
(N=139) 


Without preschool/ 
child care center d 
(N=1 45) 


Peabody Picture Vocabulary Test III 


4-11 


4-05 


4-02 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


6-00 


5-07 


5-06 


Spelling (writing) 


5-11 


5-09 


5-06 


Applied Problems (math) 


5-03 


5-01 


4-09 


Cohort 3 


PEK Cohort 3 
(N=235) 


With preschool/ 
child care center 
(N=152) 


Without preschool/ 
child care center 
(N=78) 


Peabody Picture Vocabulary Test III 


5-04 


4-08 


4-05 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


6-02 


6-00 


5-09 


Spelling (writing) 


6-04 


6-02 


5-09 


Applied Problems (math) 


5-05 


5-01 


4-09 



a In years and months. 

b Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

0 Kindergarten classmates were divided into two groups - those who attended preschool, Head Start or a child care center prior to attending kindergarten, 
and those who did not. 

d Children with missing data on preschool/child care experience were included in the no preschool/child care center group. 
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A21. PEK school component (fall 2006). Teachers’ ratings in kindergarten: PEK Cohort 1 vs. 
kindergarten classmates 



Assessment 




PEK Cohort 1 


Mean standard scores 3 

Kindergarten classmates' 3 

With preschool/ Without preschool/ 
child care center child care center 


Social Skills Rating System 
Total Social Skills d 


Mean 


103.60 


99.96 


101.03 




Adjusted mean 6 


103.61 


100.71 


99.96* 




Number assessed 


235 


139 


98 


Problem Behaviors 6 


Mean 


94.64 


95.25 


94.91 




Adjusted mean 6 


94.68 


94.34 


96.09 




Number assessed 


236 


141 


100 


Academic Competence f 


Mean 


97.14 


94.32 


88.37 




Adjusted mean 6 


96.62 


95.38 


88.06*** 




Number assessed 


221 


132 


84 



Note: includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Kindergarten classmates were divided into two groups - those who attended preschool , Head Start or a child care center prior to attending kindergarten, 
and those who did not. 

c Adjusted for gender, age, race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

d Higher scores indicate higher social skills. 

e Higher scores indicate more problem behaviors. 

f Higher scores indicate higher academic competence. 

* p< 05, compared to PEK Cohort 1. 

*** p<001, compared to PEK Cohort 1. 
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A22. PEK school component (fall 2007). Teachers’ ratings in kindergarten: PEK Cohort 2 vs. 
kindergarten classmates 



Assessment 




PEK Cohort 2 


Mean standard scores 3 

Kindergarten classmates' 3 

With preschool/ Without preschool/ 

child care center child care center 


Social Skills Rating System 
Total Social Skills 


Mean 


106.35 


100.39 


101.52 




Adjusted mean 0 


106.67 


100.79** 


100.60** 




Number assessed 


238 


119 


132 


Problem Behaviors 


Mean 


93.62 


96.42 


95.86 




Adjusted mean 0 


93.25 


96.07* 


96.85** 




Number assessed 


244 


129 


139 


Academic Competence 


Mean 


97.10 


93.79 


87.60 




Adjusted mean 0 


97.27 


94.48* 


86.66*** 




Number assessed 


242 


130 


140 



Note: includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Kindergarten classmates were divided into two groups - those who attended preschool , Head Start or a child care center prior to attending kindergarten , 
and those who did not. 

c Adjusted for gender , age, race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

* p<. 05, compared to PEK Cohort 2. 

** p<.01, compared to PEK Cohort 2. 

*** p< 001, compared to PEK Cohort 2. 
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A23. PEK school component (fall 2008). Teachers’ ratings in kindergarten: PEK Cohort 3 vs. 
kindergarten classmates 



Assessment 




PEK Cohort 3 


Mean standard scores 3 

Kindergarten classmates' 3 

With preschool/ Without preschool/ 
child care center child care center 


Social Skills Rating System 
Total Social Skills 


Mean 


104.9 


96.1 


99.0 




Adjusted mean 0 


104.5 


97.2*** 


98.0** 




Number assessed 


206 


140 


72 


Problem Behaviors 


Mean 


93.9 


99.9 


98.0 




Adjusted mean 0 


93.8 


99.3*** 


99.4** 




Number assessed 


207 


142 


74 


Academic Competence 


Mean 


98.2 


90.9 


86.0 




Adjusted mean 0 


97.8 


91.3*** 


86.2*** 




Number assessed 


205 


142 


73 



Note: includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Kindergarten classmates were divided into two groups - those who attended preschool , Head Start or a child care center prior to attending kindergarten, 
and those who did not. 

c Adjusted for gender, age, race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 
among the groups being compared. 

* p<. 05, compared to PEK Cohort 3. 

** p<.01, compared to PEK Cohort 3. 

*** p<. 001, compared to PEK Cohort 3. 
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A24. PEK school component. Academic test standard score one-year change, fall 2006 
(kindergarten) to fall 2007 (first grade): PEK Cohort 1 vs. kindergarten classmates 3 



Test 


Number 

assessed 


Mean standard scores' 3 

Kindergarten 1 st grade 

(fall 2006) (fall 2007) Change 0 


PEK Cohort 1 




Peabody Picture Vocabulary Test III 


238 


91.1 


93.4 


+2.3** 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


237 


101.8 


103.3 


+1.5* 


Spelling (writing) 


238 


102.2 


104.0 


+1.8** 


Applied Problems (math) 


237 


93.8 


102.7 


+8.9*** 


Kindergarten classmates 




Peabody Picture Vocabulary Test III 


261 


86.1 


90.0 


+3.9*** 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


259 


97.1 


100.1 


+3.0*** 


Spelling (writing) 


260 


98.3 


102.5 


+4.2*** 


Applied Problems (math) 


258 


90.1 


100.4 


+10.3*** 



a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten , they are 
followed as long as they remain in schools in Saint Paul. 

b Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. These scores are age-standardized, meaning 
that no change in scores from one year to the next indicates normative progress, positive change indicates accelerated progress, and negative change 
indicates slower progress in comparison to one’s peers. 

c Fall of first grade score minus fall of kindergarten score. 

* p<05 

** p<01 

*** p<001 
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A25. PEK school component. Academic test standard score one-year change, fall 2007 
(kindergarten) to fall 2008 (first grade): PEK Cohort 2 vs. kindergarten classmates 3 



Test 


Number 

assessed 


Mean standard scores' 3 

Kindergarten 1 st grade 

(fall 2007) (fall 2008) Change 0 


PEK Cohort 2 




Peabody Picture Vocabulary Test III 


225 


92.2 


92.3 


+0.2 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


224 


103.5 


105.4 


+1.9** 


Spelling (writing) 


224 


104.6 


105.6 


+1.0 


Applied Problems (math) 


224 


94.7 


103.9 


+9.2*** 


Kindergarten classmates 




Peabody Picture Vocabulary Test III 


214 


84.2 


87.9 


+3.8*** 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


214 


97.8 


101.4 


+3.6*** 


Spelling (writing) 


214 


99.2 


103.3 


+4.0*** 


Applied Problems (math) 


210 


90.0 


100.2 


+10.2*** 



a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten , they are 
followed as long as they remain in schools in Saint Paul. 



b Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. These scores are age-standardized, meaning 
that no change in scores from one year to the next indicates normative progress, positive change indicates accelerated progress, and negative change 
indicates slower progress in comparison to one’s peers. 

c Fall of first grade score minus fall of kindergarten score. 

* p<05 

** p<01 

*** p<001 
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A26. PEK school component. Academic test age equivalency one-year change, fall 2006 
(kindergarten) to fall 2007 (first grade): PEK Cohort 1 vs. kindergarten classmates 3 



Test 


Number 

assessed 


Mean age-equivalency scores 
(years-months) 

Kindergarten 1 st grade 

(fall 2006) (fall 2007) Change 


PEK Cohort 1 




Peabody Picture Vocabulary Test III 


238 


4-11 


6-01 


+14 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


237 


5-10 


6-11 


+13 months 


Spelling (writing) 


238 


5-09 


6-11 


+14 months 


Applied Problems (math) 


237 


5-03 


6-08 


+17 months 


Kindergarten classmates 




Peabody Picture Vocabulary Test III 


261 


4-07 


5-10 


+15 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


259 


5-07 


6-09 


+14 months 


Spelling (writing) 


260 


5-06 


6-09 


+15 months 


Applied Problems (math) 


258 


4-11 


6-08 


+21 months 



a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten, they are 
followed as long as they remain in schools in Saint Paul. 
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A27. PEK school component. Academic test age equivalency one-year change, fall 2007 
(kindergarten) to fall 2008 (first grade): PEK Cohort 2 vs. kindergarten classmates 3 



Test 


Number 

assessed 


Mean age-equivalency scores 
(years-months) 

Kindergarten 1 st grade 

(fall 2007) (fall 2008) Change 


PEK Cohort 2 




Peabody Picture Vocabulary Test III 


225 


5-00 


6-02 


+14 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


224 


6-00 


7-00 


+12 months 


Spelling (writing) 


224 


5-11 


7-00 


+13 months 


Applied Problems (math) 


224 


5-03 


6-11 


+20 months 


Kindergarten classmates 




Peabody Picture Vocabulary Test III 


214 


4-03 


5-09 


+18 months 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


214 


5-07 


6-10 


+15 months 


Spelling (writing) 


214 


5-06 


6-11 


+17 months 


Applied Problems (math) 


210 


4-11 


6-08 


+21 months 



a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten, they are 
followed as long as they remain in schools in Saint Paul. 
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A28. PEK school component (fall 2007). Academic test standard scores in first grade: PEK Cohort 
1 vs. classmates 3 



Mean standard scores' 3 



Test 




PEK 
Cohort 1 


Classmate comparison group 
in 1 st grade 0 

With preschool/ Without preschool/ 
child care center child care center 


Peabody Picture Vocabulary Test 


Mean 


93.4 


91.6 


88.7 


III 


Adjusted mean d 


93.3 


91.0 


89.3** 




Number assessed 


238 


121 


140 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


Mean 


103.2 


101.2 


98.9 




Adjusted mean d 


102.8 


102.2 


98.7** 




Number assessed 


238 


121 


140 


Spelling (writing) 


Mean 


104.0 


103.9 


101.2 




Adjusted mean d 


103.7 


104.8 


100.9* 




Number assessed 


238 


121 


140 


Applied Problems (math) 


Mean 


102.4 


100.3 


100.0 




Adjusted mean d 


102.0 


101.1 


100.0 




Number assessed 


238 


121 


140 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 



a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten , they are 
followed as long as they remain in schools in Saint Paul. 

b Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

0 The classmate comparison group was divided into two groups - those who attended preschool , Head Start , or a child care center prior to attending 
kindergarten , and those who did not. 

d Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

* p< 05, compared to PEK Cohort 1. 

** p<01, compared to PEK Cohort 1. 
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A29. PEK school component (fall 2008). Academic test standard scores in first grade: PEK Cohort 
2 vs. classmates 3 



Mean standard scores' 3 









Classmate comparison group 
in 1 st grade 0 


Test 




PEK 

Cohort 2 
(N=230-232) 


With preschool/ 
child care center 
(N=110) 


Without preschool/ 
child care center 
(N=71) 


Peabody Picture Vocabulary Test III 


Mean 


92.1 


88.9 


86.7 




Adjusted mean d 


91.1 


90.8 


86.9* 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


Mean 


105.4 


99.2 


103.5 




Adjusted mean d 


105.3 


100.5*** 


101.8* 


Spelling (writing) 


Mean 


105.7 


101.6 


104.7 




Adjusted mean d 


105.8 


102.7* 


102.8* 


Applied Problems (math) 


Mean 


104.0 


99.8 


101.3 




Adjusted mean d 


103.6 


101.2 


100.5 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 

a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten, they are 
followed as long as they remain in schools in Saint Paul. 

b Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

0 The classmate comparison group was divided into two groups - those who attended preschool, Head Start, or a child care center prior to attending 
kindergarten, and those who did not. 

d Adjusted for gender, age, race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 
differences among the groups being compared. 

* p< 05, compared to PEK Cohort 2. 

** p<.01, compared to PEK Cohort 2. 
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A30. PEK school component. Academic test age-equivalency scores in first grade: PEK students 
vs. classmates 3 

Mean adjusted 13 age-equivalency scores 
(years-months) 







Classmate comparison group 
in 1 st grade 0 


Test 


PEK 
Cohort 1 
(N=238) 


With preschool/ 
child care center 
(N=121) 


Without preschool/ 
child care center 
(N=140) 


Peabody Picture Vocabulary Test III 


6-02 


6-00 


5-10 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


6-11 


6-10 


6-08 


Spelling (writing) 


6-11 


6-11 


6-09 


Applied Problems (math) 


6-08 


6-08 


6-05 


PEK 

Cohort 2 
(N=232) 


With preschool/ 
child care center 
(N=110) 


Without preschool/ 
child care center 
(N=71) 


Peabody Picture Vocabulary Test III 


6-00 


6-00 


5-08 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 


7-00 


6-10 


6-11 


Spelling (writing) 


7-00 


6-11 


6-11 


Applied Problems (math) 


6-11 


6-08 


6-08 



a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten , they are 
followed as long as they remain in schools in Saint Paul. 

b Adjusted for gender ; age, race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

0 The classmate comparison group was divided into two groups - those who attended preschool, Head Start, or a child care center prior to attending 
kindergarten, and those who did not. 
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A31. PEK school component (fall 2007). Teachers’ ratings in first grade: PEK Cohort 1 vs. 
classmatesa 



Assessment 




PEK 
Cohort 1 


Mean standard scores' 3 

Classmate comparison group 
in 1 st grade 0 

With preschool/ Without preschool/ 
child care center child care center 


Social Skills Rating System 
Total Social Skills 


Mean 


99.9 


99.2 


103.9 




Adjusted mean d 


99.9 


100.1 


103.0 




Number assessed 


210 


108 


117 


Problem Behaviors 


Mean 


97.8 


98.0 


96.6 




Adjusted mean d 


97.9 


96.9 


97.4 




Number assessed 


211 


109 


117 


Academic Competence 


Mean 


95.4 


93.4 


91.5 




Adjusted mean d 


95.2 


93.9 


91.4** 




Number assessed 


212 


107 


118 



Note: Includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten , they are 
followed as long as they remain in schools in Saint Paul. 

b Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

c First-grade classmates were divided into two groups - those who attended preschool \ Head Start , or a child care center prior to attending kindergarten , 
and those who did not. 

d Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status. 

** p<.01, compared to PEK Cohort 1. 
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A32. PEK school component (fall 2008). Teachers’ ratings in first grade: 
PEK Cohort 2 vs. classmatesa 



Assessment 




PEK 

Cohort 2 


Mean standard scores' 3 

Classmate comparison group 
in 1 st grade 0 

With preschool/ Without preschool/ 
child care center child care center 


Social Skills Rating System 
Total Social Skills 


Mean 


101.5 


98.6 


101.4 




Adjusted mean d 


101.5 


99.2 


100.4 




Number assessed 


206 


94 


57 


Problem Behaviors 


Mean 


96.7 


101.4 


99.5 




Adjusted mean d 


96.6 


101.0* 


100.5 




Number assessed 


208 


94 


57 


Academic Competence 


Mean 


95.3 


90.8 


92.7 




Adjusted mean d 


95.4 


91.2** 


92.0 




Number assessed 


211 


94 


58 



Note: Includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a The classmate comparison group was defined as kindergarten classmates of former PEK students in the 10 PEK schools. After kindergarten , they are 
followed as long as they remain in schools in Saint Paul. 

b Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

c First-grade classmates were divided into two groups - those who attended preschool \ Head Start , or a child care center prior to attending kindergarten , 
and those who did not. 

d Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status. 

* p<.05, compared to PEK Cohort 2. 

** p<.01, compared to PEK Cohort 2. 
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A33. Kindergarten teacher ratings on connections with PEK 

Strongly 

disagree 


Somewhat 

disagree 


Somewhat 

agree 


Strongly 

agree 


Pre-kindergarten and kindergarten teachers at my school 
work together as a team in a Professional Learning 
Community. 






36% 


64% 


1 have received training and support on how to effectively 
differentiate instruction to meet the needs of a diverse 
student population. 


4% 




52% 


44% 


During the kindergarten transition period, the Pre- 
kindergarten teacher and 1 communicated about my 
students’ skills and needs (e.g., using the Pre-kindergarten 
assessment results). 


8% 


16% 


32% 


44% 


1 used student information given by the Pre-kindergarten 
teacher to help develop lessons, activities, and/or grouping 
strategies for my students this year. 


8% 


20% 


32% 


40% 


There is sufficient communication between the Pre- 
kindergarten teacher and 1. 




16% 


32% 


52% 


1 observed the Pre-kindergarten classroom this year. 


46% 


12% 


17% 


25% 


The Pre-kindergarten teacher observed my classroom this 
year. 


36% 


20% 


20% 


24% 


Kindergarten teachers at my school regularly use individual 
student assessment data to inform and tailor teaching in the 
classrooms. 




4% 


12% 


84% 
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Community-based PEK 



A34. PEK community component. Demographic characteristics of PEK community-based Cohort 1 
(2006-07) 



Home 

N Percent 


Center 

N Percent 


Age as of September 1, 2006 




3 


13 


48% 


52 


47% 


4 


14 


52% 


58 


53% 


Total 


27 


100% 


110 


100% 


Gender 




Male 


15 


56% 


50 


45% 


Female 


12 


44% 


60 


55% 


Total 


27 


100% 


110 


100% 


Free/reduced-price lunch eligibility 




Eligible 


13 


50% 


90 


87% 


Ineligible 3 


13 


50% 


13 


13% 


Total 


26 


100% 


103 


100% 


Ethnicity 




American Indian 


0 


0% 


2 


2% 


Asian 


4 


15% 


7 


6% 


Latino 


1 


4% 


19 


17% 


Black 


8 


31% 


58 


53% 


White 


13 


50% 


21 


19% 


Bi-racial or Multiracial 


0 


0% 


2 


2% 


Total 


26 


100% 


109 


100% 


Home language 




English 


27 


100% 


94 


85% 


Hmong 


0 


0% 


4 


4% 


Spanish 


0 


0% 


8 


7% 


Other 


0 


0% 


4 


4% 


Total 


27 


100% 


110 


100% 
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A34. PEK community component. Demographic characteristics of PEK community-based Cohort 1 
(2006-07) (continued) 







Home 




Center 




N 


Percent 


N 


Percent 


Received Special Education services 




Yes 


2 


8% 


3 


3% 


No 


23 


92% 


99 


97% 


Total 


25 


100% 


102 


100% 


In target population 13 




Yes 


14 


54% 


94 


91% 


No 


12 


46% 


9 


9% 


Total 


26 


100% 


103 


100% 



a Includes families who were not eligible for free or reduced-price lunch as well as families who did not apply. 

b Child is in one or more of the following categories: eligible for free or reduced-price lunch , ELL, or receives Special Education services. 

Notes: Because children in the first three child care cohorts were not assessed in fall of their PEK year, child care Cohorts 1, 2, and 3 reflect all children 

attending PEK child care. In contrast, school-based cohorts are defined as PEK students who were assessed in fall of their PEK year. 
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A35. PEK community component. 
(2007-08) 


Demographic characteristics of PEK commu 

Home 

N Percent 


nity-based Cohort 2 
Center 

N Percent 


Age as of September 1, 2007 




3 


17 


49% 


42 


53% 


4 a 


18 


51% 


37 


47% 


Total 


35 


100% 


79 


100% 


Gender 




Male 


17 


49% 


42 


57% 


Female 


18 


51% 


32 


43% 


Total 


35 


100% 


74 


100% 


Free/reduced-price lunch eligibility 




Eligible 


9 


27% 


56 


92% 


Ineligible 13 


24 


73% 


5 


8% 


Total 


33 


100% 


61 


100% 


Ethnicity 




American Indian 


0 


0% 


2 


3% 


Asian 


2 


6% 


4 


6% 


Latino 


7 


20% 


6 


8% 


Black 


7 


20% 


47 


64% 


White 


19 


54% 


14 


19% 


Total 


35 


100% 


73 


100% 


Home language 




English 


32 


91% 


67 


92% 


Hmong 


0 


0% 


3 


4% 


Spanish 


3 


9% 


3 


4% 


Total 


35 


100% 


73 


100% 


Received Special Education services 




Yes 


2 


6% 


0 


0% 


No 


30 


94% 


60 


100% 


Total 


32 


100% 


60 


100% 
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A35. PEK community component. Demographic characteristics of PEK community-based Cohort 2 
(2007-08) (continued) 



Home 





N 


Percent 


N 


Percent 


In target population 0 




Yes 


14 


40% 


58 


73% 


No 


21 


60% 


21 


27% 


Total 


35 


100% 


79 


100% 



Center 



a One child who was 5 years old as of September 1, 2007 , is included in the 4-year-old group. 
b Includes families who were not eligible for free or reduced-price lunch as well as families who did not apply. 



c Child is in one or more of the following categories: eligible for free or reduced-price lunch , ELL, or receives Special Education services. 

Notes: Because children in the first three child care cohorts were not assessed in fall of their PEK year, child care Cohorts 1, 2, and 3 reflect all children 

attending PEK child care. In contrast, school-based cohorts are defined as PEK students who were assessed in fall of their PEK year. 
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A36. PEK community component. Demographic characteristics of PEK community-based Cohort 3 
(2008-09) 



Home 

N Percent 


Center 

N Percent 


Age as of September 1, 2008 




3 


16 


44% 


68 


46% 


4 a 


20 


56% 


79 


54% 


Total 


36 


100% 


147 


100% 


Gender 




Male 


22 


61% 


71 


48% 


Female 


14 


39% 


76 


52% 


Total 


36 


100% 


147 


100% 


Free/reduced-price lunch eligibility 




Eligible 


17 


53% 


97 


69% 


Ineligible 13 


15 


47% 


44 


31% 


Total 


32 


100% 


141 


100% 


Ethnicity 




American Indian 


2 


6% 


3 


2% 


Asian 


1 


3% 


2 


1% 


Latino 


1 


3% 


19 


13% 


Black 


14 


42% 


79 


54% 


White 


15 


45% 


43 


29% 


Total 


33 


100% 


146 


100% 


Home language 




English 


29 


81% 


137 


94% 


Russian 


4 


11% 


- 


- 


Spanish 


1 


3% 


7 


5% 


Other 


2 


6% 


2 


1% 


Total 


36 


100% 


146 


100% 


Received Special Education services 




Yes 


2 


6% 


5 


3% 


No 


32 


94% 


140 


97% 


Total 


34 


100% 


145 


100% 
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A36. PEK community component. Demographic characteristics of PEK community-based Cohort 3 
(2008-09) (continued) 



Home 





N 


Percent 


N 


Percent 


In target population 0 




Yes 


23 


72% 


102 


73% 


No 


9 


28% 


38 


27% 


Total 


32 


100% 


140 


100% 



Center 



a One child who was 5 years old as of September 1, 2008 , is included in the 4-year-old group. 
b Includes families who were not eligible for free or reduced-price lunch as well as families who did not apply. 



c Child is in one or more of the following categories: eligible for free or reduced-price lunch , ELL, or receives Special Education services. 

Notes: Because children in the first three child care cohorts were not assessed in fall of their PEK year, child care Cohorts 1, 2, and 3 reflect all children 

attending PEK child care. In contrast, school-based cohorts are defined as PEK students who were assessed in fall of their PEK year. 
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A37. PEK community component. PEK community-based Cohort 1 children’s attendance 
(September 1, 2006, to August 31, 2007) 



Number of days present 


N 


Home 

Percent 


N 


Center 

Percent 


Age 3 




Fewer than 60 days 


- 


- 


- 


- 


60-80 


- 


- 


3 


6% 


81-100 


- 


- 


6 


12% 


101-120 


- 


- 


3 


6% 


121-140 


2 


15% 


4 


8% 


141-160 


1 


8% 


5 


10% 


161-180 


2 


15% 


6 


12% 


181-200 


3 


23% 


7 


13% 


201-220 


4 


31% 


6 


12% 


More than 220 days 


1 


8% 


12 


23% 


Total 


13 


100% 


52 


100% 


Average 




182 


168 


Median 




184 


178 


Range 




121-239 


65-241 


Age 4 




Fewer than 60 days 


- 


- 


2 


3% 


60-80 


- 


- 


1 


2% 


81-100 


- 


- 


5 


9% 


101-120 


1 


7% 


2 


3% 


121-140 


2 


14% 


3 


5% 


141-160 


4 


29% 


11 


19% 


161-180 


3 


21% 


12 


21% 


181-200 


1 


7% 


9 


16% 


201-220 


2 


14% 


4 


7% 


More than 220 days 


1 


7% 


9 


16% 


Total 


14 


100 


58 


100% 


Average 




163 


165 


Median 




161 


175 


Range 




111-235 


38-248 



Note: The number of days offered by family child care homes varied widely , with some homes not participating in PEK during this entire period. The 

range was 129 to 252 days between September 1, 2006 , and August 31, 2007. For child care centers, it was 250 to 253 days. 
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A38. PEK community component. PEK community-based Cohort 2 children’s attendance 
(September 1, 2007, to April 30, 2008) 



Number of days present 


N 


Home 

Percent 


N 


Center 

Percent 


Age 3 




Fewer than 60 days 


2 


12% 


3 


7% 


60-80 


- 


- 


6 


14% 


81-100 


2 


12% 


11 


26% 


101-120 


1 


6% 


2 


5% 


121-140 


3 


18% 


3 


7% 


141-160 


9 


53% 


12 


29% 


161-180 


- 


- 


5 


12% 


181-200 


. 


- 


- 


- 


201-220 


- 


- 


- 


- 


More than 220 days 


- 


- 


- 


- 


Total 


17 


100% 


42 


100% 


Average 




125 


114 


Median 




141 


116 


Range 




40-159 


37-165 


Age 4 




Fewer than 60 days 


- 


- 


4 


11% 


60-80 


2 


11% 


4 


11% 


81-100 


2 


11% 


1 


3% 


101-120 


- 


- 


4 


11% 


121-140 


3 


17% 


5 


14% 


141-160 


11 


61% 


17 


46% 


161-180 


- 


- 


2 


5% 


181-200 


- 


- 


- 


- 


201-220 


- 


- 


- 


- 


More than 220 days 


- 


- 


- 


- 


Total 


18 


100% 


37 


100% 


Average 




134 


122 


Median 




151 


144 


Range 




70-158 


20-164 



Notes: In 2007-08 , attendance was recorded for both centers and homes from September 1, 2007 through April 30, 2008. Some of the family child 

care programs did not offer PEK during this entire period, however. The number of months offered by family child care homes ranged from six to eight 
months during this period. 
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A39. PEK community component. PEK community-based Cohort 3 children’s attendance 
(September 1, 2008, to August 31, 2009) 



Number of days present 


N 


Home 

Percent 


N 


Center 

Percent 


Age 3 




60-80 


1 


6% 


- 


- 


81-100 


1 


6% 


- 


- 


101-120 


- 


- 


2 


3% 


121-140 


3 


19% 


6 


9% 


141-160 


2 


12% 


4 


6% 


161-180 


4 


25% 


11 


16% 


181-200 


2 


12% 


7 


10% 


201-220 


2 


12% 


12 


185 


More than 220 days 


1 


6% 


26 


38% 


Total 


16 


100% 


68 


100% 


Average 




159 


198 


Median 




168 


206 


Range 




72-225 


107-249 


Age 4 




60-80 


- 


- 


1 


1% 


81-100 


- 


- 


3 


4% 


101-120 


- 


- 


2 


3% 


121-140 


1 


5% 


3 


4% 


141-160 


3 


15% 


6 


8% 


161-180 


4 


20% 


14 


18% 


181-200 


9 


45% 


13 


16% 


201-220 


3 


15% 


15 


19% 


More than 220 days 


- 




22 


28% 


Total 


20 


100% 


79 


100% 


Average 




181 


192 


Median 




185 


197 


Range 




132-216 


78-249 



Note: The range in the number of days offered at family child care homes was 171 to 251 days between September 1, 2008, and August 31, 2009, 

with the exception of one child care home that offered 87 days. For child care centers, it was 247 to 253 days. 
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A40. PEK community component (fall 2007). Achievement test standard scores in kindergarten: 
PEK community-based Cohort 1 vs. kindergarten classmates 

Standard score 3 



Test 


PEK 

community-based 
Cohort 1 


Kindergarten classmates' 3 
With preschool/ 

child care center Without preschool/ 
(not PEK) child care center 


Peabody Picture Vocabulary Test III 




Mean 


97.4 


85.7 


83.1 


Adjusted mean 0 


93.0 


87.1* 


83.2** 


Number assessed 


47 


139 


145 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 




Mean 


102.0 


98.0 


96.3 


Adjusted mean 0 


101.3 


98.9 


95.7* 


Number assessed 


47 


139 


145 


Spelling (writing) 




Mean 


103.0 


99.9 


97.2 


Adjusted mean 0 


99.5 


101.0 


97.2 


Number assessed 


47 


139 


145 


Applied Problems (math) 




Mean 


96.0 


91.4 


87.9 


Adjusted mean 0 


92.8 


92.7 


87.7 


Number assessed 


47 


139 


140 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Upon kindergarten entry , PEK community-based Cohort 1 children were compared to PEK schooi-based Cohort 2 (Figure A21) as well as the PEK 
school-based Cohort 2 comparison group (presented here). The comparison group was divided into two groups - those who attended preschool , Head 
Start , or a child care center (other than PEK) prior to attending kindergarten , and those who did not. Children with missing data on preschool/child care 
experience were included in the no preschool/child care center group. 

0 Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, Special Education status, and test date 

differences among the groups being compared. 

*p<.05, compared to PEK community-based Cohort 1, **p<.01, compared to PEK community-based Cohort 1. 
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A41. PEK community component (fall 2008). Achievement test standard scores in kindergarten: 
PEK community-based Cohort 2 vs. kindergarten classmates 

Standard score 3 



Test 


PEK 

community-based 
Cohort 2 


Kindergarten classmates' 3 
With preschool/ 

child care center Without preschool/ 
(not PEK) child care center 


Peabody Picture Vocabulary Test III 




Mean 


101.5 


87.6 


81.1 


Adjusted mean 0 


93.1 


88.1 


83.6** 


Number assessed 


34 


152 


78 


Woodcock-Johnson Tests of Achievement III 




Letter-Word Identification (reading) 




Mean 


105.9 


102.3 


98.4 


Adjusted mean 0 


101.8 


103.2 


98.3 


Number assessed 


34 


152 


78 


Spelling (writing) 




Mean 


111.3 


105.1 


101.1 


Adjusted mean 0 


107.7 


106.2 


100.4** 


Number assessed 


34 


152 


78 


Applied Problems (math) 




Mean 


97.7 


90.1 


84.6 


Adjusted mean 0 


93.1 


91.1 


84.7** 


Number assessed 


34 


152 


78 



Note: Significance tests were conducted based on a directional hypothesis that former PEK children scored higher than each of the two classmate 

groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Upon kindergarten entry , PEK community-based Cohort 2 children were compared to PEK school-based Cohort 3 (Figure A28) as well as the PEK 
school-based Cohort 3 comparison group (presented here). The comparison group was divided into two groups - those who attended preschool , Head 
Start , or a child care center (other than PEK) prior to attending kindergarten, and those who did not. 

0 Adjusted for gender, age, race/ethnicity, free/reduced-prlce lunch eligibility, English Language Learner status, Special Education status, and test date 
differences among the groups being compared. 

* p<.05, compared to PEK community-based Cohort 2. 

** p<.01, compared to PEK community-based Cohort 2. 
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A42. PEK community component (fall 2007). Achievement test standard scores in kindergarten: 
PEK community-based Cohort 1 vs. PEK school-based Cohort 2 

Standard score 3 



Test 


PEK 

community-based 
Cohort 1 


PEK 

school-based 
Cohort 2 


Peabody Picture Vocabulary Test III 


Mean 


97.4 


92.1 


Adjusted mean b 


92.7 


92.9 


Number assessed 


47 


266 


Woodcock-Johnson Tests of Achievement III 


Letter-Word Identification (reading) 


Mean 


102.0 


103.1 


Adjusted mean b 


100.4 


103.4 


Number assessed 


47 


266 


Spelling (writing) 


Mean 


103.0 


104.1 


Adjusted mean b 


100.9 


104.5 


Number assessed 


47 


266 


Applied Problems (math) 


Mean 


96.0 


94.5 


Adjusted mean b 


92.8 


95.1 


Number assessed 


47 


266 



Note: There were no statistically significant differences in adjusted mean test scores between the two groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibiiity, English Language Learner status , Special Education status , and test date 

differences between the groups being compared. 
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A43. PEK community component (fall 2008). Achievement test standard scores in kindergarten: 
PEK community-based Cohort 2 vs. PEK school-based Cohort 3 

Standard score 3 



Test 


PEK 

community-based 
Cohort 2 


PEK 

school-based 
Cohort 3 


Peabody Picture Vocabulary Test III 


Mean 


101.5 


94.6 


Adjusted mean b 


95.9 


95.4 


Number assessed 


34 


235 


Woodcock-Johnson Tests of Achievement III 


Letter-Word Identification (reading) 


Mean 


105.9 


106.9 


Adjusted mean b 


105.3 


107.0 


Number assessed 


34 


234 


Spelling (writing) 


Mean 


111.3 


110.0 


Adjusted mean b 


109.7 


110.2 


Number assessed 


34 


234 


Applied Problems (math) 


Mean 


97.7 


96.7 


Adjusted mean b 


95.1 


97.1 


Number assessed 


34 


234 



Note: There were no statistically significant differences in adjusted mean test scores between the two groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibiiity, English Language Learner status , Special Education status , and test date 

differences between the groups being compared. 
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A44. PEK community component (fall 2007). Teacher ratings of social skills, problem behaviors, 
and academic competence in kindergarten: PEK community-based Cohort 1 vs. kindergarten 
classmates 



Assessment 




PEK 

community-based 
Cohort 1 


Standard score 3 

Kindergarten classmates' 3 

With preschool/ Without 

child care center preschool/ 

(not PEK) child care center 


Social Skills Rating System 
Total Social Skills 


Mean 


98.4 


100.4 


101.5 




Adjusted mean 0 


99.3 


101.2 


100.6 




Number assessed 


38 


119 


132 


Problem Behaviors 


Mean 


103.3 


96.4 


95.9 




Adjusted mean 0 


101.4 


95. 7 d 


97.0 




Number assessed 


38 


129 


139 


Academic Competence 


Mean 


93.7 


93.8 


87.6 




Adjusted mean 0 


93.9 


94.5 


86.8** 




Number assessed 


38 


130 


140 



Note: Includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Upon kindergarten entry , PEK community-based Cohort 1 children were compared to PEK school-based Cohort 2 (Figure A48) as well as the PEK 
school-based Cohort 2 comparison group (presented here). The comparison group was divided into two groups - those who attended preschool , Head 
Start , or a child care center (other than PEK) prior to attending kindergarten , and those who did not. Children with missing data on preschool/child care 
experience were included in the no preschool/child care center group. 

0 Adjusted for gender , age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

d If we had used a non-directional hypothesis instead of a directional hypothesis, then this result would have been statistically significant (p<0.05) 
compared to PEK community-based Cohort 1. 

* p< 05, compared to PEK community-based Cohort 1. 

** p<.01, compared to PEK community-based Cohort 1. 
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A45. PEK community component (fall 2007). Teacher ratings of social skills and problem behaviors 
in kindergarten: Subscale results for PEK community-based Cohort 1 vs. kindergarten 
classmates 









Raw scores 








Kindergarten classmates 3 


Assessment 




PEK 

community-based 
Cohort 1 


With preschool/ 
child care center 
(not PEK) 


Without 
preschool/child 
care center 


Total Social Skills subscales 










Cooperation 


Mean 


13.6 


14.8 


14.6 




Adjusted mean b 


14.3 


15.0 


14.2 




Number assessed 


36 


120 


130 


Assertion 


Mean 


12.9 


11.9 


12.5 




Adjusted mean b 


13.1 


12.1 


12.2 




Number assessed 


36 


124 


130 


Self-control 


Mean 


13.2 


14.3 


14.8 




Adjusted mean b 


13.7 


14.5 


14.5 




Number assessed 


34 


121 


134 


Problem Behaviors subscales 








Externalizing 


Mean 


2.8 


1.4 


1.5 




Adjusted mean b 


2.3 


1.4° 


1.7 




Number assessed 


37 


128 


140 


Internalizing 


Mean 


2.0 


2.0 


1.6 




Adjusted mean b 


1.8 


1.9 


1.7 




Number assessed 


37 


130 


139 


Hyperactivity 


Mean 


5.6 


3.3 


3.4 




Adjusted mean b 


5.0 


3.1 d 


3.7 




Number assessed 


38 


126 


139 



Note: Includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors subscales) than each of the two classmate groups. 



a Upon kindergarten entry , PEK Cohort 1 child care children were compared to PEK school-based Cohort 2 (Figure A49) as well as the PEK school-based 

Cohort 2 comparison group (presented here). The comparison group was divided into two groups - those who attended preschool \ Head Start , or a child 
care center (other than PEK) prior to attending kindergarten , and those who did not. Children with missing data on preschool/child care experience were 
included in the no preschool/child care center group. 

b Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

c If we had used a non-directional hypothesis instead of a directional hypothesis, then this result would have been statistically significant (p<0.05) compared 
to PEK community-based Cohort 1. 

d If we had used a non-directional hypothesis instead of a directional hypothesis, then this result would have been statistically significant (p<0.01) compared 
to PEK community-based Cohort 1. 

*p<.05, compared to PEK community-based Cohort 1, **p<.01, compared to PEK community-based Cohort 1. 
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A46. PEK community component (fall 2008). Teacher ratings of social skills, problem behaviors, 
and academic competence in kindergarten: PEK community-based Cohort 2 vs. kindergarten 
classmates 









Standard score 3 








Kindergarten classmates' 3 


Assessment 




PEK 

community-based 
Cohort 2 


With preschool/ 
child care center 
(not PEK) 


Without 

preschool/ child 
care center 


Social Skills Rating System 










Total Social Skills 


Mean 


99.7 


96.1 


99.0 




Adjusted mean 0 


100.2 


97.0 


97.1 




Number assessed 


27 


140 


72 


Problem Behaviors 


Mean 


105.7 


99.9 


98.0 




Adjusted mean 0 


102.9 


99.3 


100.2 




Number assessed 


27 


142 


74 


Academic Competence 


Mean 


95.6 


90.9 


86.0 




Adjusted mean 0 


93.9 


91.2 


85.9** 




Number assessed 


27 


142 


73 



Note: Includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors) than each of the two classmate groups. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Upon kindergarten entry , PEK community-based Cohort 2 children were compared to PEK school-based Cohort 3 (Figure A50) as well as the PEK 
school-based Cohort 3 comparison group (presented here). The comparison group was divided into two groups - those who attended preschool , Head 
Start , or a child care center (other than PEK) prior to attending kindergarten , and those who did not. 

c Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

** p<.01, compared to PEK community-based Cohort 2. 
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A47. PEK community component (fall 2008). Teacher ratings of social skills and problem behaviors 
in kindergarten: Subscale results for PEK community-based Cohort 2 vs. kindergarten 
classmates 









Raw scores 








Kindergarten classmates 3 


Assessment 




PEK 

community-based 
Cohort 2 


With preschool/ 
child care center 
(not PEK) 


Without 

preschool/ child 
care center 


Total Social Skills subscales 










Cooperation 


Mean 


14.3 


12.9 


14.1 




Adjusted mean b 


14.7 


13.1 


13.6 




Number assessed 


26 


139 


70 


Assertion 


Mean 


13.1 


11.9 


11.9 




Adjusted mean b 


12.4 


12.1 


11.7 




Number assessed 


26 


138 


70 


Self-control 


Mean 


13.5 


13.2 


14.1 




Adjusted mean b 


13.9 


13.4 


13.6 




Number assessed 


25 


139 


71 


Problem Behaviors subscales 








Externalizing 


Mean 


3.3 


2.1 


1.7 




Adjusted mean b 


2.6 


1.9 


2.2 




Number assessed 


27 


141 


74 


Internalizing 


Mean 


2.2 


2.1 


2.3 




Adjusted mean b 


2.0 


2.1 


2.4 




Number assessed 


26 


142 


74 


Hyperactivity 


Mean 


5.7 


4.3 


3.7 




Adjusted mean b 


5.1 


4.1 


4.1 




Number assessed 


26 


141 


74 



Note: Includes only students who were assessed on both social and academic skills. Significance tests were conducted based on a directional 

hypothesis that former PEK children scored higher (lower for Problem Behaviors subscales) than each of the two classmate groups. There were no 
statistically significant differences in adjusted mean test scores between PEK community-based Cohort 2 and either of the two comparison groups. 

a Upon kindergarten entry , PEK Cohort 2 child care children were compared to PEK school-based Cohort 3 (Figure A51) as well as the PEK school- 

based Cohort 3 comparison group (presented here). The comparison group was divided into two groups - those who attended preschool, Head Start, 
or a child care center (other than PEK) prior to attending kindergarten, and those who did not. 

b Adjusted for gender, age, race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 
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A48. PEK community component (fall 2007). Teacher ratings of social skills, problem behaviors, 
and academic competence in kindergarten: PEK community-based Cohort 1 vs. PEK school- 
based Cohort 2 







Standard score 3 


Assessment 




PEK 

community-based 
Cohort 1 


PEK 

school-based 
Cohort 2 


Social Skills Rating System 








Total Social Skills 


Mean 


98.4 


106.4 




Adjusted mean b 


98.7 


106.3** 




Number assessed 


38 


238 


Problem Behaviors 


Mean 


103.3 


93.6 




Adjusted mean b 


101.9 


93.8*** 




Number assessed 


38 


244 


Academic Competence 


Mean 


93.7 


97.1 




Adjusted mean b 


93.1 


97.2 




Number assessed 


38 


242 



Note: Includes only students who were assessed on both social and academic skills. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Adjusted for gender , age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

** p<.01, compared to PEK community-based Cohort 1. 

*** p<.001, compared to PEK community-based Cohort 1. 
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A49. PEK community component (fall 2007). Teacher ratings of social skills and problem behaviors 
in kindergarten: Subscale results for PEK community-based Cohort 1 vs. PEK school-based 
Cohort 2 



Assessment 


Raw scores 

PEK 

community-based 
Cohort 1 


PEK 

school-based 
Cohort 2 


Total Social Skills subscales 








Cooperation 


Mean 


13.6 


16.0 




Adjusted mean 3 


14.0 


15.9** 




Number assessed 


36 


235 


Assertion 


Mean 


12.9 


13.7 




Adjusted mean 3 


12.6 


13.7 




Number assessed 


36 


232 


Self-control 


Mean 


13.2 


15.5 




Adjusted mean 3 


13.5 


15.4* 




Number assessed 


34 


236 


Problem Behaviors subscales 








Externalizing 


Mean 


2.8 


1.4 




Adjusted mean 3 


2.5 


1 .4** 




Number assessed 


37 


243 


Internalizing 


Mean 


2.0 


1.4 




Adjusted mean 3 


2.0 


1.4 




Number assessed 


37 


243 


Hyperactivity 


Mean 


5.6 


2.7 




Adjusted mean 3 


5.1 


2.8*** 




Number assessed 


38 


243 



Note: Includes only students who were assessed on both social and academic skills. 

a Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

* p<.05, compared to PEK community-based Cohort 1. 

** p<.01, compared to PEK community-based Cohort 1. 

*** p<.001, compared to PEK community-based Cohort 1 . 
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A50. PEK community component (fall 2008). Teacher ratings of social skills, problem behaviors, 
and academic competence in kindergarten: PEK community-based Cohort 2 vs. PEK school- 
based Cohort 3 







Standard score 3 


Assessment 




PEK 

community-based 
Cohort 2 


PEK 

school-based 
Cohort 3 


Social Skills Rating System 








Total Social Skills 


Mean 


99.7 


104.9 




Adjusted mean b 


101.5 


104.7 




Number assessed 


27 


206 


Problem Behaviors 


Mean 


105.7 


93.9 




Adjusted mean b 


103.6 


94.1** 




Number assessed 


27 


207 


Academic Competence 


Mean 


95.6 


98.2 




Adjusted mean b 


97.3 


97.9 




Number assessed 


27 


205 



Note: Includes only students who were assessed on both social and academic skills. 

a Standard scores have a mean of 1 00 and a standard deviation of 15 in the national normative sample. 

b Adjusted for gender , age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

** p<.01, compared to PEK community-based Cohort 2. 
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A51. PEK community component (fall 2008). Teacher ratings of social skills and problem behaviors 
in kindergarten: Subscale results for PEK community-based Cohort 2 vs. PEK school-based 
Cohort 3 



Assessment 


Raw scores 

PEK 

community-based 
Cohort 2 


PEK 

school-based 
Cohort 3 


Total Social Skills subscales 








Cooperation 


Mean 


14.3 


16.0 




Adjusted mean 3 


15.2 


15.9 




Number assessed 


26 


204 


Assertion 


Mean 


13.1 


13.1 




Adjusted mean 3 


12.6 


13.1 




Number assessed 


26 


203 


Self-control 


Mean 


13.5 


15.1 




Adjusted mean 3 


14.3 


15.0 




Number assessed 


25 


202 


Problem Behaviors subscales 








Externalizing 


Mean 


3.3 


1.3 




Adjusted mean 3 


2.8 


1.3** 




Number assessed 


27 


207 


Internalizing 


Mean 


2.2 


1.9 




Adjusted mean 3 


2.2 


1.9 




Number assessed 


26 


206 


Hyperactivity 


Mean 


5.7 


2.4 




Adjusted mean 3 


5.1 


2.5*** 




Number assessed 


26 


205 



Note: Includes only students who were assessed on both social and academic skills. 

a Adjusted for gender ; age , race/ethnicity, free/reduced-price lunch eligibility, English Language Learner status, and Special Education status differences 

among the groups being compared. 

** p<.01, compared to PEK community-based Cohort 2. 

*** p<.001, compared to PEK community-based Cohort 2. 
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A52. PEK community component: Language and Literacy supports in Cohort 3 child care centers, 
fall 2008 to spring 2009 



ELLCO indicator and possible points for each indicator 


2008 

Average 

score 

(n=12) a 


2009 

Average 

score 

(n=12) a 


Change in average 
score and percent 
out of possible 
points 


Literacy Environment Checklist (41) 


19.3 


33.6 


+14.3 


(+35%) 


Book area (3) 


1.8 


2.8 


+1.0 


(+33%) 


Book selection (8) 


6.5 


7.8 


+1.3 


(+16%) 


Book use (9) 


1.9 


7.6 


+5.7 


(+63%) 


Book Subscale (20) 


10.2 


18.3 


+8.1 


(+41%) 


Writing materials (8) 


5.4 


7.7 


+2.3 


(+29%) 


Writing around the room (13) 


3.7 


7.7 


+4.0 


(+31%) 


Writing Subscale (21) 


9.1 


15.3 


+6.2 


(+30%) 


Classroom Observation (General Classroom Environment 
Subscale and Language, Literacy & Curriculum Subscale) 




General Classroom Environment Subscale (30) 


19.1 


24.0 


+4.9 


(+16%) 


Organization of the classroom (5) 


3.6 


4.7 


+1.1 


(+22%) 


Contents of the classroom (5) 


2.8 


4.3 


+ 1.5 


(+30%) 


Presence/use of technology (5) 


1.5 


2.4 


+0.9 


(+18%) 


Opportunities for child choice and initiative (5) 


3.5 


4.7 


+1.2 


(+24%) 


Classroom management strategies (5) 


4.0 


3.9 


-0.1 


(-2%) 


Classroom climate (5) 


3.8 


4.1 


+0.3 


(+6%) 


Language, Literacy, & Curriculum Subscale (40) 


21.8 


33.2 


+11.4 


(+29%) 


Oral language facilitation (5) 


3.0 


3.6 


+0.6 


(+12%) 


Presence of books (5) 


3.0 


4.8 


+1.8 


(+36%) 


Approaches to book reading (5) 


3.0 


5.0 


+2.0 


(+40%) 


Approaches to children’s writing (5) 


2.7 


4.6 


+1.9 


(+38%) 


Approaches to curriculum integration (5) 


2.6 


4.3 


+1.7 


(+34%) 


Recognizing diversity in the classroom (5) 


2.9 


3.6 


+0.7 


(+14%) 


Facilitating home support for literacy (5) 


2.0 


3.7 


+1.7 


(+34%) 


Approaches to assessment (5) 


2.5 


3.7 


+1.2 


(+24%) 


Literacy Activities Rating Scale (12) 


4.3 


9.5 


+5.2 


(+43%) 



a Only classrooms with pre and post assessments are included . 

Source: Classroom observations conducted by Center for Early Education and Development , University of Minnesota. 
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A53. PEK community component: Language and Literacy supports in Cohort 3 child care homes, 
fall 2008 to spring 2009 



CHELLO indicator and possible points for each indicator 


2008 
Average 
score 
N=1 1 


2009 
Average 
score 
N=1 1 


Change in average 
score and percent 
out of possible 
points 


Literacy Environment Checklist (26) 


16.9 


22.5 


+5.6 


(+22%) 


Book Area (5) 


4.1 


4.4 


+.3 


(6%) 


Book Use (9) 


6.8 


8.1 


+1.3 


(+14%) 


Writing Materials (6) 


3 


5.5 


+2.5 


(42%) 


Toys (3) 


1.8 


2.6 


+.8 


(+27%) 


Technology (3) 


1.2 


1.9 


+.7 


(+23%) 


Group/Family Observation: Physical Environment (15) 


11.3 


12.2 


+.9 


(+6%) 


Organization of the Environment (5) 


3.6 


3.8 


+.2 


(+4%) 


Materials in the Environment (5) 


3.8 


4.3 


+.5 


(+10%) 


Daily Schedule (5) 


3.9 


4.3 


+.4 


(+8%) 


Group/Family Observation: Support for Learning( 1 5) 


11.4 


12.3 


+.9 


(+6%) 


Adult Affect (5) 


3.8 


4 


+.2 


(+4%) 


Adult-Child Language Interaction (5) 


3.7 


4.4 


+.7 


(+14%) 


Adult Control Behaviors (5) 


3.8 


3.9 


+.1 


(+2%) 


Group/Family Observation: Adult Teaching Strategies (35) 


19.7 


28.4 


+8.7 


(+25%) 


Vocabulary Building (5) 


3.1 


3.9 


+.8 


(+16%) 


Responsive Strategies (5) 


3.2 


3.9 


+.7 


(+14%) 


Use of Print (5) 


2.2 


4.2 


+2 


(+4%) 


Storybook/Storytelling activities (5) 


3.9 


4.8 


+.9 


(+18%) 


Writing/Drawing activities (5) 


2.8 


4.4 


+1.6 


(+32%) 


Monitoring children’s progress (5) 


1.5 


3.4 


+1.9 


(+38%) 


Family support and interaction (5) 


3 


3.9 


+.9 


(+18%) 



Source: Classroom observations conducted by Center for Early Education and Development , University of Minnesota. 
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A54. Impacts of PEK professional development 











Number of participants 




How much do you agree or disagree 
with the following statements: 


Group 


N 


Strongly 

agree 


Somewhat 

agree 


Somewhat 

disagree 


Strongly 

disagree 


Participation in PEK professional 
development has had a large impact on 
practices at my child care center. 


Directors 


9 


8 


1 


- 


- 


Participation in PEK professional 


Center teachers 


19 


12 


7 


- 


- 


development has had a large impact on 
my teaching practices. 


Family providers 


13 


13 


- 


- 


- 



A55. Preparation of children for school 



How much do you agree or disagree 
with the following statement: 


Group 


N 


Strongly 

agree 


Number of participants 

Somewhat Somewhat 
agree disagree 


Strongly 

disagree 


My center better prepares children for 
school because of our participation in 
PEK. 


Directors 


9 


9 


- 


- 


- 


1 better prepare children for school 
because of my participation in PEK. 


Center teachers 


19 


12 


6 


1 


- 


Family providers 


13 


12 


1 


- 


- 
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